Take a close look at the map, then you will see that your X is located in a U-shaped section of the corridor. It looks like this:
The Ns mark the tiles with the 180° spinner. There two 180°-spinning-tiles of the puzzle, which are about
either preventing you from entering this section (when you come from the beginning temple-grounds and first enter this section, so the first 180° spinner is there to keep you away from the deeper grounds of the temple!)
or keeping you in the U-section since you get into an endless spinning around and walking about, until you react to the 180° spinning properly ...