I absolutely agree with your comment.

The media ALWAYS tends to report the negative things only, leaving a image to the public that SMAP is no longer popular etc.
I had seen a video in youtube about kids that are borned in 1998 and 1996 commenting about different musician, when they see SMAP, they screamed and one of the boy said that 'people that are above SMAP are problely Gods'. I used to thought that SMAP is not that popular in the young generation because of the stupid news, and this proved to me that SMAP really REALLY is a national idol group ( and that no one should believe to the stupid media - -; )
I also agree with you saying that the agency is just milking ARASHI's popularity, here is the evidence:

SMAP - Have been active for 27 years, released 54 singles.
ARASHI - Have been active for 16 years, released 47 singles.

This shows how the agency is trying so hard to push ARASHI to earn themselves more money while they are at peak. This not only affect ARASHI, but also its fans. (Who can afford for so much singles in a year T.T)

It is so true that the agency can't control SMAP at all, to be honest, I always think that the agency put 5 of them together because they are the most problematic kids. ( It's their charm! XD) One article suggest that a lot of SMAP fans think that although they are from Johnnys, they have nothing like Johnnys, a lot of fans even said that they disllike Johnnys except for SMAP. ( Well, they WERE treated badly by the people from Johnnys, scary senpais, disappointed staff because their first single didn't reached oricon #1 etc.)

And yes, I do like ARASHI, but like you, i don't think ARASHI is ever able to surpass SMAP (not like they have the need to, like why? to be honest)

SMAP is a legend, they break the image of what people see idols as in the past ( Some people actually believe that idols never poops in the past lol), they participate in variety, comedy shows, dressing up in really silly costumes doing really silly things. Kimura Kun was so successful in dramas that he is known all over the world, Nakai kun is the first idol who become MC successfully, now becoming one of the most respected MC in the variety world.

In my opinion I think what ARASHI is missing is that their experience working with famous comedian. Which is what SMAP managed to do, Kimura having a close relationship with Sanma San, Hamada San ( Downtown), and Nakai having a close relationship with Matsumodo San (DT), Tsurube San. ( Tamori san is basically SMAP's dad lol )

(BTW, hardcore ARASHI fans are really scary T.T, kind of having a complicated feeling about SMAP working with ARASHI because of my trauma towards them T.T)