People keep thinking "new Sony handheld" means a Vita 2 aka something with its own dedicated library of games. That's not what this is. If Sony are making a handheld this is what I think they'd do:
1: RDNA 2 or 3 GPU with at least base PS4 TF performance
2: 6nm process
3: 8 GB HBM2 for " gaming" model (HAS to provide PS4 memory bandwidth with small physical footprint and low power usage; HBM2 is only option for this).
4: 12 GB (8 GB HBM2, 4 GB LPDDR5) for "phone" model.
5: 512 GB embedded Flash storage for "gaming" model; 1 TB for "phone" model. Both can use additional external storage.
The main purpose of this device would be to serve as a 100% native portable PS4. Every single PS4 game, runs with no issue and gains portability as a result. However, since it shares GPU & CPU architecture with PS5, it can essentially just straight up leverage PS5 APIs and dev environment.
That opens up many possibilities for smaller devs and games that may still want to target the PS4 install base, but have the luxury of their software being easily available to PS5 owners and PS4 owners, in addition to owners of the handheld. This would be especially appealing to Japanese and indie devs who want a mainstream portable with more power than a Switch (and possibly Switch 2) to target, with common API tools and the brand strength of PlayStation, as well as knowing their games can be played on this handheld, PS4 & PS5 consoles.
There should be two versions, both with the same baseline specs (CPU, GPU, memory bandwidth, storage type), but otherwise customized to the "lower" end dedicated portable gaming market, and the other targeted at the "higher" end smartphone market.
This would affect certain design choices. For example, the "phone" model would need to dual-boot into PS OS and Android; the "gaming" model would only need PS OS. The "gaming" model would not need a high-quality 16 megapixel front and rear camera setup, whereas the "phone" model definitely needs that sort of thing. The "phone" model would need SIM card support, whereas the "gaming model does not.
In the same way, the " gaming" model needs expected physical button and trigger inputs by default built into its shell, probably as a modular fit-to-form design. The " phone" model doesn't need this by default; instead one could just purchase that type of controller component separately.
I think the only real complication in this could be in the event where a company like Sony wants to make a smaller mobile game say based on a classic PS IP, then make that game available on the PS devices, because they'd need to make two versions. Well, maybe. I think they could in fact just need to make one but build two different executables, and optimize them.