Cite this problem as Problem 8.
The relative entropy of entanglement is an entanglement monotone that quantifies to what extent a given state can be operationally distinguished (in the sense of Stein’s Lemma) from the closest state which is either separable or has a positive partial transpose (PPT). For a state it is defined as 
where stands for the convex sets of separable or PPT states, and is the quantum relative entropy. The problem is to find a closed formula for this quantity for systems consisting of two qubits.
The interpretation of the relative entropy of entanglement is a geometrical one: it is related to the error probability with which a state is mistakenly assumed to be merely classically correlated or PPT in quantum hypothesis testing. This entanglement monotone is an upper bound to the distillable entanglement, and in its asymptotic version conjectured to be identical to the Rains’ bound for distillable entanglement. As most other monotones of entanglement, and all other known monotones that are provably asymptotically continuous, the actual evaluation of this quantity amounts to solving an optimization problem. In the case at hand, it is a convex optimization problem.
The entanglement of formation is a monotone which is also defined as an optimization problem. If it turned out that the entanglement of formation was in fact additive (see problem 7), then this quantity could be interpreted as the entanglement cost, which fleshes out the resource character of entanglement. Historically, it was very important that for systems consisting of two qubits, the entanglement of formation can (quite astonishingly) be evaluated: the Wootters formula  is a closed formula for the entanglement of formation for two-qubit systems. The proof exploits a number of the particular properties that are available for two-qubit systems  — and only for them. The task is to explicitly solve the convex optimization problem posed by the relative entropy of entanglement.
Ref.  presents the solution to a related problem: for a two-qubit system, given a state on the boundary of separable states , it characterizes the states for which .
Last results have been obtained in  and . More information on the partial results will follow.
 V. Vedral and M.B. Plenio, Phys. Rev. A 57, 1619 (1998).
 W. Wootters, Phys. Rev. Lett. 78, 5022 (1997).
 K.G.H. Vollbrecht, and R.F. Werner, J. Math. Phys. 41, 6772 (2000).
 S. Ishizaka, Phys. Rev. A 67, 060301(R) (2003).
 A. Miranowicz and S. Ishizaka, Closed formula for the relative entropy of entanglement, Phys. Rev. A, 78. 032310 (2008) and arXiv:0805.3134v3
 S. Friedland and G. Gour, Closed formula for the relative entropy of entanglement in all dimensions, arXiv:1007.4544v2