The solenoid is easy to check. Disconnect both small wires. If the solenoid is still "stuck", the solenoid is defective.
If the solenoid is no longer stuck, check the voltage at the the wire from the ignition switch. It should be dead unless the key is turned to the "start" position. See page 9
here for the circuit.