Comment by AnthonyMouse

First you solve the chicken and egg problem by writing a bootstrap compiler for the language in some other language (e.g. a C# compiler in C or C++, or a C compiler in assembly language). Then you can compile programs in the new language, and compilers are programs, so you can write a C# compiler in C# and the binary so produced will be able to compile its own source.