In theoretical physics, BRST quantization (where the BRST refers to Becchi, Rouet, Stora and Tyutin) is a relatively rigorous mathematical approach to quantizing a field theory with a gauge symmetry. Quantization rules in earlier QFT frameworks resembled "prescriptions" or "heuristics" more than proofs, especially in non-abelian QFT, where the use of "ghost fields" with superficially bizarre properties is almost unavoidable for technical reasons related to renormalization and anomaly cancellation. The BRST supersymmetry was introduced in the mid-1970s and was quickly understood to justify the introduction of these Faddeev–Popov ghosts and their exclusion from "physical" asymptotic states when performing QFT calculations. Work by other authors a few years later related the BRST operator to the existence of a rigorous alternative to path integrals when quantizing a gauge theory.
Only in the late 1980s, when QFT was reformulated in fiber bundle language for application to problems in the topology of low-dimensional manifolds, did it become apparent that the BRST "transformation" is fundamentally geometrical in character. In this light, "BRST quantization" becomes more than an alternate way to arrive at anomaly-cancelling ghosts. It is a different perspective on what the ghost fields represent, why the Faddeev–Popov method works, and how it is related to the use of Hamiltonian mechanics to construct a perturbative framework. The relationship between gauge invariance and "BRST invariance" forces the choice of a Hamiltonian system whose states are composed of "particles" according to the rules familiar from the canonical quantization formalism. This esoteric consistency condition therefore comes quite close to explaining how quanta and fermions arise in physics to begin with.
In certain cases, notably gravity and supergravity, BRST must be superseded by a more general formalism, the Batalin–Vilkovisky formalism.
Read more about BRST Quantization: Technical Summary, Gauge Transformations in QFT, Mathematical Approach To BRST, The BRST Operator and Asymptotic Fock Space, Gauge Bundles and The Vertical Ideal, BRST Formalism