Pulled my exhaust manifolds on my 2001 6.0 to put on some headers and noticed this:



Is this just bad casting or is this a build up? Can I do anything about it?

 

Unless your rig has about 2 miles on it, that is completely normal. A little soot is on every exhaust port out there.
That is just the difference in the head port and the gasket and manifold. In a perfect world, everything would be port matched, but obviously this is not a perfect world. Manufacturing cost would go through the roof if some one had to port match everything that goes out of the factory.

 

Your 317 casting heads have "D" shaped exhaust ports while the manifold opening is round, so you will notice carbon buildup on the outside of the head where the opening to the manifold does not covered. As mentioned above, carbon is a byproduct of combustion, totally normal.