....just a friend who has some special permits...; "friends with rights",
That reminds me of a girl who once told me she was spending the weekend with a guy and when I asked if he was her boyfriend she said "No, he's just a friend with privileges."
I might be getting in over my head here because I'm an older man. Traditionally, a sweetheart is the "special someone", someone you are in love with and they could be your boyfriend (your only boyfriend), your fiancé or your husband. It is a term of endearment.
In the case of your boyfriend, I suppose you could have more than one and still have one of them as your "sweetheart" but to have more than one suggests that you are fickle and therefore the term sweetheart wouldn't mean much.
To further muddy the water, people often call their children or even strangers "sweetheart" much like they do in Spanish when they call someone "mi amor" There was an old TV series called "Kojak" in which the lead character called everybody sweetheart, even other men.