And for that matter, there are some substantial security threats external to the device as well;
1) That you are registered with them using your real phone number. This information is something that they should absolutely NOT have access to, since it can be used both to establish that you have communicated with certain other signal users, and WHEN,
2) While server code is supposedly public, the actual servers used are not auditable. You don't know if the code they're running is the same as what is public, and therefore you don't know if vulnerabilities have been added.
3) If had gms installed when setting it up, it feeds message notifications through google, which is an untrusted 3rd party that also doesn't need to know about messages you're receiving, even if the contents are encrypted.
4) Hostile governments can use the FACT that you are using certain encrypted communications providers as evidence that you are engaging in anti-government hostility. Often they respond by doing things like freezing your bank accounts and branding you racists and misogynists with unacceptable views.