In the Southwest during the Civil War, a mysterious stranger, Joe (Clint Eastwood), and a Mexican outlaw, Tuco (Eli Wallach), form an uneasy partnership -- Joe turns in the bandit for the reward money, then rescues him just as he is being hanged. When Joe's shot at the noose goes awry during one escapade, a furious Tuco tries to have him murdered. The men re-team abruptly, however, to beat out a sadistic criminal and the Union army and find $20,000 that a soldier has buried in the desert.

The story develops around two plot lines set in the Old West town of "Flagstone." First, a man dubbed "Harmonica" seeks revenge against the outlaw, Frank. Second, Frank works as a hired gun for railroad tycoon, Morton, who is trying to acquire certain land owned by the Brett McBain family. A second outlaw, Cheyenne, also figures into the story.

Harmonica kills three men who ambush him on his arrival at the train station. The dusters they wear lead him to believe they are Cheyenne's men. Meanwhile, Frank and his henchmen kill Brett McBain and his three children at their ranch, which they call "Sweetwater." Frank leaves behind evidence to frame Cheyenne for the murders.