Dynamic Plugin Reload & Loggers

I've noticed an issue (that I may be misdiagnosing), that it seems that if I try and use a logger in my plugin, the plugin can't be reloaded. When I look at the memory snapshot, it looks like the my PluginClassLoader is reachable from "thisClassLoader" of "org.apache.commons.logging.LogFactory", if I never load the Logger then I don't have this issue. I'm using the org.slf4j and relying on the ones already on the classpath (I don't have one included as a plugin dependency), not sure if that matters.