child custody of mother

I badly need advice about the custody of my kids. I'm separated with my husband for 6 years now. The kids are with him for 2 years and 5 months now. We had a written agreement about it. Now i want to get the kids back and transfer them to a much nearer school where i live. But he said we need to revoke the agreement first and i think he doesnt want to give my kids back under my custody. He is living with his mistress with the kids. I know that my kids are having a hard time gettimg along with his partner. What should i do. Will that agreement be a basis that i abandoned my kids and gave him the full custody of my children?what case can i file to get my kids back? Hoping for your immediate response. Thank you.