No, I don't think that was the major problem. It wasn't good, but it's a relatively minor problem compared to other problems.
By the time the Swimsuit Issue started doing that (mid-2010s or so), it was already irrelevant in popular culture and had been since ~2000. Once more people started getting high speed internet and access to more porn online, the very tame Swimsuit Issue became less of a cultural phenomenon.
Sports Illustrated was a lot more than the Swimsuit Issue. For most of its history, Sports Illustrated published weekly, so there were 50 other issues a year about sports. Sports Illustrated was too slow to react to the internet era and didn't become an internet-focused media outlet in the late 1990s/early 2000s. Print journalism has been dying due to internet and SI did not handle that change well. ESPN.com beat them to the internet and ESPN.com made itself an internet destination faster. ESPN as a company is having problems due to people no longer subscribing to cable or satellite TV service but ESPN.com is useful.
It has difficult for a lot of the legacy print journalism publications to succeed in the internet era. This has been true for both newspapers and magazines.