Not having `unsafe_no_drop_flag` caused some weirdness with optimizing buffers
and bytes. For now, remeove it.