Some creatures have a fly speed. Some creatures with a fly speed also have the ability to hover specified.

The PHB, in Chapter 9: Combat, Movement and Position, Flying Movement says:

Flying creatures enjoy many benefits of mobility, but they must also deal with the danger of falling. If a flying creature is knocked prone, has its speed reduced to 0, or is otherwise deprived of the ability to move, the creature falls, unless it has the ability to hover or it is being held aloft by magic, such as by the fly spell.

The conditions in this paragraph that would result in the creature falling have to do with having movement involuntarily removed or restricted. That might suggest that if you don't have "hover" in your flying speed, you can't hover.

But there is a difference between being knocked unconscious, or frozen, or webbed, or otherwise rendered unable to move, and choosing to hover, by flapping your wings, or standing on your jets, or whatever.

So, can flying creatures choose to hover, even if they don't have hover in their flying speed?

A creature with a Flying Speed is not required to use their movement to stay aloft unless a feature specifically says so

It might be best to demonstrate this by showing an obvious counter-example.

Consider, for example, the Totem Warrior subclass of the Barbarian Class, who at level 14, is given the option of gaining a flight speed while raging:

Eagle. While raging, you have a flying speed equal to your current walking speed. This benefit works only in short bursts; you fall if you end your turn in the air and nothing else is holding you aloft.

—Path of the Totem Warrior, Player's Handbook, pg. 50

So for this specific situation, the Barbarian would gain a flying speed—but also gain the stipulation that this speed cannot keep them aloft at the end of their turn.

Conversely, most creatures that have Flying speeds have no such restriction or stipulation: they simply specify a Flying speed of X', without this kind of text. That means that they would not fall if they stop moving, or if they cease to move during their turn, unless they were subjected to one of the conditions specified in your original post and lacked the Hover feature.

The conditions in this paragraph that would result in the creature falling have to do with having movement involuntarily removed or restricted. That might suggest that if you don't have "hover" in your flying speed, you can't hover.

This suggests that if you don't have "hover" in your flying speed, you will fall when your movement is involuntarily removed or restricted. It makes no suggestions on what will happen if you voluntarily don't move.

The conditions given (speed 0, prone, deprived of movement) are given without ambiguity, so can be considered a complete list of what would make a flying creature fall. As such, every creature with a fly speed can choose not to move on their turn and stay aloft (because doing so is not on the list of conditions that would cause a fall).

Flying creatures can remain aloft unless their rules says otherwise

From the Flying Movement section under Movement and Position (PHB, p. 191) we have (emphasis mine):

Flying creatures enjoy many benefits of mobility, but they must also deal with the danger of falling. If a flying creature is knocked prone, has its speed reduced to 0, or is otherwise deprived of the ability to move, the creature falls, unless it has the ability to hover or it is being held aloft by magic, such as by the fly spell.

Possessing the 'hover' ability prevents a creature from falling when knocked prone or deprived of movement, but otherwise has no effect on where the creature may end its turn. By default creatures that possess a fly speed may end their turn in the air. However, doing so without the 'hover' ability exposes them to falling damage should an enemy somehow deprive them of movement.

Some creatures or abilities may limit your flying ability and require you to land at the end of your turn. See Do I have to land at the end of my turn? for evidence that this is not the general case. One example of a feature with this restriction is the level 14 Path of the Totem Warrior (Eagle) feature which says (emphasis mine):

While raging, you have a flying speed equal to your current walking speed. This benefit works only in short bursts; you fall if you end your turn in the air and nothing else is holding you aloft.

If the feature allowing you to fly does not possess text to a similar effect, you can end your turn in the air or remain stationary while aloft. Having the 'hover' ability merely negates the risk of falling.

Yes

All it requires to stay in the air without falling is to have a flying speed. If your flying speed is reduced to zero you fall unless you have hover.

Your interpretation is correct.