Not sure if this is the right way to do it but it works everytime for me: Twist the knobs 'til they are loose (as you have done) and then hook a finger under them and pull up (w/ some force). Yes, it pops them out on an angle, but there is no fidgeting and (so far) hasn't broken anything.
Hope it's working better. One of the fasteners on my storage box (the right one) twists and pulls right out just as you would expect. The other one isn't so cooperative. I fiddled with it for an eternity 'til I decided to employ a bit more finger-leverage. Works every time now. However, if I find a less persuasive method I'll gladly switch techniques.