The capture button refers to a button near the bottom of the left Joy-Con, as well as on the Nintendo Switch Pro Controller. It allows the player to capture pictures of the Switch game they are playing and view them in the system's capture gallery, where they can then be edited with customizable text or be posted to either Facebook or Twitter.
If the button is pressed, the Switch takes a screenshot of whatever game the player is playing, but if the button is held, the Switch saves the last 30 seconds of gameplay as a video, though the latter feature often varies by game. Captured videos can also be edited and posted in a similar fashion.
Usages in the Super Smash Bros. series
Likely due to the in-game replay feature, Ultimate is one of the few games on the Nintendo Switch that does not have the ability to save videos via the capture button. Normal screenshots can still be taken, though given the return of the series' trademark camera, such a function is arguably redundant, as pictures taken via in-game methods are saved to the Switch's capture gallery in the same fashion to regular screenshots.