Truth about higher octane gas vs low octane?

During my high school years kids were always saying that putting higher octane fuel in your car will give it more horsepower. I always disagreed with them. Higher octane gas is made for cars with higher fuel compression. With a higher fuel compression the gas can be less flammable. I was told by an engineer that putting a lower octane gas then recommended in a car will increase horsepower. However it might create spark knock that is not good for the engine if it was made for high octane fuels. When your car doesn't need a higher octane and you put one in are you just waisting your money and actually loosing performance? Do you agree or disagree with me that higher octane, if the engine is not made for it you get less performance?