He was held captive on a British ship in the battle of Baltimore which was a naval siege on an american fortress. That's when he wrote the star spangled banner. He was referring to american flag on the fortress. All the Americans had to do was lower it and the siege would stop and they would surrender. But they kept the flag up all night.
Hope this helps
