the ignition is in the off position right? If so it sounds as if there is a wire either arching over making the solenoid receive power to the output side. It could also be a faulty solenoid even if new it may have fused together or stuck on.
I would unhook the output side(wire going to starter) for solenoid and test for power there. If no power check your wiring because it is getting juice form a HOT wire somwhere sending power to your starter.