In the medical and healthcare fields, the demand for dialogue systems capable of empathetic responses is increasing, while conventional chatbots face challenges in adjusting responses according to the consultant’s psychological state. This study aims to develop a voice agent that estimates users’ intentions and emotional attitudes in real-time and optimizes response content and tone, along with validating its effectiveness. In the initial stage, we designed an empathetic and flexible dialogue agent by controlling response strategies based on emotion estimation results and reflecting them in the LLM generation process. In this presentation, we report on the results of this investigation.