[DanEC]

I assume you have a vacuum assist brake booster. If the booster is working and holding vacuum properly, after running the engine and shutting it down, you should then be able to apply and hold the brakes 3 or 4 times before the pedal eventually goes soft and starts to sink towards the floor. If when you first apply the brakes and the pedal after a moment starts sinking to the floor the booster is not functioning properly.

And as stated your engine may just not produce enough vacuum with a hot cam for proper operation.