From a game design “formalist” point of view, they are not different. A rules-centric view of games doesn’t care whether the interface is computerized, mediated via apparatus, or physical, so it makes no distinction between computer chess and physical chess; similarly, it makes no distinction between the rules of, say, baseball, implemented within a computer or by players on a field. They’re both still recognizably baseball. You can diagram them; you can port the higher level rules between media; you can implement even a phsyical version with a ruleset that requires everyone to play on their knees, or in wheelchairs.
The major distinction arises with subgames and interfaces present within the rules. For example, baseball-the-sport makes use of extensive implementations of physics, thanks to the real world providing a very robust physics engine. It also has a very rich set of subgames regarding mastering the controls of the human physical body. Computerized baseball is relatively limited on that front, mostly requiring mastery of just your hands as they manipulate the controller.
Sports historically refers to physical games, but of course even many non-sport games have large physical components involving either strength or dexterity. Many children’s games, such as jacks and tiddlywinks immediately come to mind (not that jacks was always a children’s game…).