Email - send with SendGrid and receive with IMAP


I’m trying to setup an email account that uses SendGrid for outgoing and IMAP for incoming. Ideally I would use this email with HelpDesk. The problem I have is that when I have the “Use different Email ID” box checked and use the API key in the password field for sending with SendGrid I get the error “Please check your email login credentials”. Is there a way to have different credentials for sending a receiving? I imagine I need separate credentials for both IMAP and send grid. Does anyone have any other workarounds?