Unambiguous discrimination of mixed states: A description based on system-ancilla coupling

We propose a general description on the unambiguous discrimination of mixed states according to the system-environment coupling, and present a procedure to reduce this to a standard semidefinite programming problem. In the two states case, we introduce the canonical vectors and partly simplify the problem to the case of discrimination between pairs of canonical vectors. By considering the positivity of the two by two matrices, we obtain a series of new upper bounds of the total success probability which depends on both the prior probabilities and specific state structures.