Unable to start WebDriverAgent session because of xcodebuild failure

Getting the below error message when running appium cli inside a bitrise agent on a simulator

Request failed with status 500 due to An unknown server-side error occurred while processing the command. Original error: Unable to start WebDriverAgent session because of xcodebuild failure: An unknown server-side error occurred while processing the command. Original error: Could not proxy command to the remote server. Original error: connect ECONNREFUSED 127.0.0.1:8100

I am not even building an app. I have a built app attached to my repo, that I am trying to test for the proof of concept.

Config :
platformName: ‘iOS’,
deviceName: ‘iPhone 8 Plus’,
platformVersion: ‘14.4’,
automationName: ‘XCUITest’,

OS - Catalina
Xcode - 12.4
Appium Version - 1.19.0