Historically, via surveying which relied on triangulation.
Let's say you have one known point of reference in the entire world, your "Point A". You place a metal marker there so you can keep track of where it is. Then, you can make use of the core surveyor tool a "theodolite". This is a device which is in essence very simple, it's just a telescope on an "alt-az" mount (up/down and left/right) on a tripod. The important bit about a theodolite is that it has markings for the direction (azimuth or compass bearing) and the height (altitude or elevation) the telescope is pointed in relative to the tripod.
So, let's say you have a theodolite, how do you use it? First things first, you place it on your known point of reference. Second, you need to align it so that the "North" for the theodolite is actual, true North. So you need a compass (or you can wait until night fall and use Polaris). Then you need to level the theodolite, so the top of the tripod is sitting flat. Now your theodolite's measurements will be accurate. For example, if you were to use the theodolite simply to look at Polaris then it should have an azimuth of exactly due North with an elevation equal to your latitude.
One point isn't much value, so how do you add more reference points? Well, if you use the theodolite to sight in on some likely candidate reference point you can get the bearing and elevation of that location. Then, if you take a measuring tape or use one of those little wheel based distance measurers and walk over to the new location (taking your theodolite with you) you can figure out how far it is to that location.
Now, you know what the angle of elevation was from your first point to this new Point B, and now you know the distance along the ground. Using some simple trigonometry you can determine the altitude difference between the two points as well. Now you can place Point B in a 3D grid relative to Point A. You know the compass bearing direction that Point B is relative to A and you know the total distance from A to B. From that, again using trigonometry, you can work out the North/South distance between A and B as well as the East/West distance, so you can precisely locate B on a map relative to A. And now you know the relative altitude as well!
So, let's carry this forward. You can add a bunch more of these points just by iterating on this process. And you can eventually have points which go out to the coast and you can get a look at a sea level marker that has a mark for the average sea level and from that determine the absolute altitude of your Point A (and all other points) relative to "sea level". So now you can continue to do this sort of thing even across continents that aren't connected by land.
But what if you want to find the position of a point you can't get to, like the top of a mountain? Well, all you really need are two known points that are near it. You take a theodolite reading from each one and then you can find out where on the map the top of the mountain is. You have a sight line (at a specific compass heading) from one point and then you have a sight line from another point, where they intersect is where the mountain is. The more separation you have in those starting points the closer the two sight lines will be to perpendicular vs. parallel and the less error there will be in where the lines intersect. Anyway, once you know where the lines intersect you then automatically know how far away the mountain is from both of your original points, and that means you can determine the height of the mountain using trigonometry and the angle of elevation of the sightings of the mountain from each point. And, of course, you can use this trick to expand out your network of "known points" without having to physically measure off the distances between each and every point, making it much easier to do so.
Today we can instead use GPS, satellite RADAR, and so forth, but surveying techniques were how we originally determined the heights of mountains.