the 3ds was not faltering, the system sold around the same as PSP and 360. Nintendo was doing OK at the time.
Moreover, Nintendo has seen far more success with their home systems than Sony or Valve have seen with their portable systems. The Wii U sold more than pretty much any non-nintendo handheld sans PSP.
If they made a no-frills regular powerful box home system, they'd do fine because that was their business model up until Wii.
Most of their failure comes from doing weird shit that didn't mesh well with home systems, i.e minidiscs, cartridges, weird controllers, game censorship, alienating third parties, etc. Modern Nintendo isn't like that, they wouldn't make the same mistakes.