With respect to the mismatch problem caused by the color differences of the same person in different cameras, a brightness transfer function space probability estimation based human tracking algorithm with multi-cameras in the lαβ color space is presented. The algorithm firstly trains the extracted human objects in a low dimension to acquire the brightness transfer function sets. Then the probability density function is modeled utilizing the principal component analysis to the sub-space of the brightness transfer function. Finally, matching probability between two persons which are in one camera and the other is gotten by using the probability density function. Reliable human match is gotten by calibrating the color of pixels on the base of the brightness transfer function. Experimental results are reliable and robust, and they show that the method provides an efficient way to realize human tracking with multi-cameras without respecting overlapping or non-overlapping field of view. Better results are gotten by the brightness transfer function sets in the lαβ color space than that in the RGB color space or without the brightness transfer function sets.