Yep, if only I had access to apk 
I have to get to already installed app on the device.
Okay, figured it out by following steps listed on this site: https://sqa.stackexchange.com/questions/12897/how-to-do-a-testing-on-an-already-installed-android-application-using-appium-s
These are the caps that allowed me to start my app:
{
"platformName": "Android",
"deviceName": "emulator-5554",
"platformVersion": "11.0",
"orientation": "PORTRAIT",
"automationName": "UiAutomator2",
"noReset": true,
"newCommandTimeout": 0,
"appPackage": "com.myapp.android.alpha",
"appActivity": "com.myapp.android.ui.activity.LoginActivity"
}
Finally made it!