As described in Encounters, the boundary of when an encounter ends and another begins is at the discretion of the narrator. That being said, there are plenty of natural boundaries that make sense to use in the context of a greater story. Typically, an encounter should be considered one "scene" - it may or may not involve danger, and further, it may have moments that are not in danger and then moments that are.
As an example, an encounter could involve arriving at a lord's manor for a dinner party. Throughout the encounter, the PCs and NPCs socialize and converse and are fed. The PCs take a moment to sneak away to investigate suspicious activities in the manor, breaking into a locked room on the top floor. As they discover the incriminating evidence, the lord and his guards burst in. A fight ensues, but the PCs are outmanned and must flee. Eventually, their pursuers lose sight of them in the deep woods. This entire experience could be considered one encounter.
Part of the art of the narrator is to understand when an encounter break feels right for the story, but also to be aware of the limited resources that players have at their disposal that are dependent on encounter changes. Many actions or other talented abilities have "once per encounter" (or some other number) limitations imposed; by making an encounter too long and too dangerous, these talents will be exhausted and the players may find themselves without a leg to stand on.