I highly recommend Symfonium. It has the queue behavior you're looking for and is feature-rich across the board. I use it for listening to music hosted on a jellyfin server, but it's compatible with many different providers as well as local files.
It's a one time $4.99 purchase, but for what you get it's a great deal. You can even purchase directly from the dev if you're looking to avoid Google Play altogether, which I'm sure will appeal to many on this forum.