Many, many materials have been used to make sandpaper. As discussed before, ground glass was one of the early commercial grits, which is why the English still refer to "abrasive sheets" as "Glasspaper".
Aluminum Oxide Sandpaper
Today, there are a number of standard materials used to make sandpaper. Perhaps the most commonly used to make sandpaper, and most likely what you will find at hardware stores and home centers, is aluminum oxide sandpaper. It is commonly used for sanding wood and metal. Garnet can also be found although it is not as common. It is traditionally preferred for woodworking and you may hear older woodworkers refer to "garnet paper".
Silicon Carbide Sandpaper
Silicon Carbide Sandpaper is pretty common as well. You'll recognize it as the black "wet/dry" sandpaper, with a plastic sheet backer. Very fine grades of sandpaper may use chromium oxide, usually in 600 grit or finer.
The sheets shown above are Aluminum Oxide (tan), Garnet (red), and Silicone Carbide (black).