Serious Eats - seriouseats.com

Slice them, and cook them with some orange juice, a little sugar and a splash of Grand Marnier for a fruit compote to go really well with a citrus or chocolate cake. (You can add a little cornstarch to thicken if needed.)
If you don't want to use it over cake, you can process it smooth and fold it onto a buttercream, or whipped cream/creme fraiche as a cake icing. Good folded into mousse this way also.

Use them to infuse vodka.

Chop them and add them to a muffin or cake batter with walnuts/almonds.

Toss sliced kumquats in a sweet fruit salad marinated in booze, or lime, etc. for a contrast.

Slice and bake over chicken or fish with some OJ and soy sauce and honey.
OR-do the same but with lemon juice, honey and lots of smoky cumin.

Or, like me, just eat them whole!

I candied some and didn't like them anywhere near as much as raw. Now I feel like I'll be wasting them if I do anything but eat them raw. I do enjoy getting friends who have never tried them to share some with me, though. They're always weirded out at first and then enjoy them just as much as I do. They're a lot of fun to eat!

Slice them thin and add them (raw) to salad. They are zippy little tidbits. Really great if you're using a semi-sweet vinaigrette and/or including goat cheese in your salad.