Your question dives into a grey area for this forum.. As its practice is to not talk about building any machine that could run OSX.. That said, other then BIOS changes, a Mac is just another Computer or Personal Computer. Except Apple designs the system, out sources many of the parts to companies like Foxconn who make standard computer parts like motherboards your mentioned.
Now if we go by your heading questions: "What's The most advanced motherboard that someone can build themselves?" Technically speaking, what ever is out today. ASUS ROG series is pretty cutting edge motherboards, technically anyone can make a board that runs like those. However if one where to make it say from scratch by stopping by a 1980's Radio Shack. It would look pretty hideous and huge. Most modern motherboards have about 8 layers of circuitry smashed into that board, cutting down on size and any wiring. But yea, you can make one, but it may look like a fire hazard waiting to happen.
Motherboards are not some magical item, they are simply a circuit board with a BIOS, Northbridge controller, south bridge controller, hand full of voltage regulators and resisters, plus all your accessories that are controlled by your south bridge controller chip, like USB, built in sound, PCI, etc, and so on.. Then slap on a few RAM slots, CPU, AGP or PCI-X slots that are handled by the north bridge and bam.. Got yourself a motherboard..
That said, technically anyone could make a modern motherboard. But it takes practice and loads of testing to ensure its stable and not cook anything you plug into it. This is why everyone just buys them.