Part of my problem is the spring inside the brake drum does not pull the works back to the rest position very well.
I think you can(should?) fit an external brake return spring to the brake arm, the brake light switch in your picture is fairly common and readily available, there are a others which work in the same way which you could try.
I think you can(should?) fit an external brake return spring to the brake arm, the brake light switch in your picture is fairly common and readily available, there are a others which work in the same way which you could try.