So you upgraded to WordPress 5.8 which is good. But now in media library instead of infinity scroll there is button Load more and message Showing 40 of x media items. This is frustating because you have to press this button Load more every time just to get next fourty media items.
To get rid of button Load more and return infinity scrolling to media library you have to add one hook
Add this line of code to your theme functions.php
Thanks to WordPress team, they implemented hook which can revert back to infinity scroll
So as of WordPress 5.8 default media library behaviour is button Load more
I dont know, by maybe they could have added this as a checkbox in Media Settings panel
There is also plugin to solve this problem. This plugin only ads hook and nothing more