AuthenticationException when establishing UserEndpoint

We have build a UCMA bot which automatically responds to user input.

For this application we log in with an Active Directory UC enabled account. In the past we have tested this with our support account (<Domainname>\support) which logs in correctly.

private void EndpointEstablishCompleted(IAsyncResult result)
{
    UserEndpoint userEndpoint = result.AsyncState as UserEndpoint;
    
    try
    {
        userEndpoint.EndEstablish(result);

        Console.WriteLine("Done");

    }
    catch (Exception exception)
    {
        _log.Fatal("Listener setup failed: " + exception.Message);
    }
}

However, using a couple of other Active Directory accounts we get an exception after:

userEndpoint.EndEstablish(result);

Exception:

AuthenticationException

Not authorized to perform the requested operation, request is refused.

The users which get this exception are UC enabled users, they can login through the lync client.

We've also tested this with a newly created account, logging in through this account however does NOT throw an exception.

I can't seem to find the cause why some of these accounts can log in and some not. They have the same permissions.

Does anyone have a solution for this problem?