When compiling V-USB for attiny85 clocked at 16MHz using PLL (internal oscillator), the device was not recognized with “Device not accepting address” error messages in the kernel log.
The problem was that setting clocks in V-USB to 16000 (16MHz) only works with a crystal oscillator. When using internal oscillator, this setting does not work, and you need to set 16.5Mhz frequency by using the following in usbconfig.h
#define USB_CFG_CLOCK_KHZ 16500
After recompilation the device recognized properly, and worked correctly.
Hopefully this would save you a few hours of investigation.