Don't use the HSI clock, that's not specced for USB use. Also, drop the unnecessary delays that were working around old bugs.
Don't use the HSI clock, that's not specced for USB use. Also, drop the unnecessary delays that were working around old bugs.