Doesn't look like there's currently a way to delete invoices and/or tickets.
At this stage this isn't possible but we'll be looking into this as part of a future iteration, as we receive feedback.
Hope this helps.
+1 to wanting the ability to delete/cancel registrations.
Perhaps tangentially related to this, you can currently delete the Registration form responses, but it doesn't have any affect on the event registration itself. It may be useful to have the form submissions and event registrations more tightly linked.
I'll add your support to this request, and pass along the feedback about having the two linked more closely.
Hope this helps,