I like @wx9's define for niche, and I agree that the term FOSS in here may not be accurate and might not even be OP's thoughts. (Thank you for the elaboration, @nonsuch)

In my personal opinion, it's the device/service that does not follow the trend and mainstream, or follow what mega company / organisation / or even gov, want you to be.

In my guess, the intention of OP could be:

Hardware wise:
- Has certain degree of freedom of components control / driver (Purism, PinePhone)
- Has certain characters that against the trend of mainstream modern phones (Fxtec Pro1)
- Has freedom to choose which OS to run, and not locked to a certain OS

Software wise:
- Has true multitasking like desktop, unlike Android and iOS
- Has a certain level of openness, user can easily tweak and customise and play/kill the OS without breaking many rules
- Has a slightly better transparency of what the OS does or if it collects info of where you are what you searched or even what you say
- Not having millions of services and free games that occupy your time and make mind ******ed.......

Unfortunately nothing is perfect in this world, even our beloved N900 / N950 is not completely FOSS. But the most important is there are people trying and there are users to support the effort swim against the stream.