Does Appium interfere with custom URL authentication callback on Android?

Support
1

Our hybrid app is using OAuth2 as authentication scheme. The login button in our app forwards our customers to an external website, which is provided by an identity server and which is shown via in-app browser. On this website the customers enter their login credentials. The identity server verifies the credentials, generates a user access token and issues a callback to our app. Once the app receives the callback, it brings its UI in foreground again and switches to the logged-in state. The callback has the following format:

nameofourapp://auth/callback

This URL is based on a custom URL scheme which tells the device OS to launch our app.

In manual testing the login flow works perfectly fine on both iOS and Android. However, in our automatic Appium tests, the login flow works in the iOS case only. It does not work for Android. More specifically, in the Android case the Appium test manages to push the login button in the app, enter the user credentials on the external website, press the login button on that site but then the callback never reaches the app (we verified this by looking at the app logs). So maybe the callback does not happen at all or (what I think is more likely) there is something interfering with it so that it does not have the wanted effect, namely, to open our app again via custom URL scheme.

Does anybody have an idea what could go wrong here? Can it be that Appium somehow interferes with the authentication callback? Or that it interferes with the handling of custom URLs on the device? Why does the problem only show for Android and not for iOS?

We execute our Appium tests at Bitbar devices in the cloud. Could it be that the Bitbar somehow interferes with the authentication callback or the handling of custom URLs? I don’t think so because in manual live testing sessions at Bitbar the login flow works fine also on Android.

Below I paste our Appium code that we use for the login, just in case it should be relevant:

public LoginPage pressLoginBtnInApp() {

       click(btnLoginSignup);
       waitSeconds(5,true);
           
      boolean bChromeView = true;
      String sContext = TestUtils.setDriverContext(driver, bChromeView);
      TestUtils.log(Level.INFO, logger, false, "Changed context to: " + sContext);
      waitSeconds(5,true);
                 
      String sURL = driver.getCurrentUrl();
      TestUtils.log(Level.INFO, logger, false, "Available URL: " + sURL);
      driver.get(sURL);
             
      return new LoginPage(driver, page);
  }
      
public BasePage pressLoginBtnInAuthWebsite() {      
      click(btnLogin);
      waitSeconds(10, false);
    
      boolean bChromeView = false;
      TestUtils.setDriverContext(driver, bChromeView);
      waitSeconds(5, true);
             
      return page;
   }

public static String setDriverContext(AppiumDriver<MobileElement> driver, boolean bChromeView) {
    String sWebViewContext =  "";
    Set<String> contextNames = driver.getContextHandles();
    TestUtils.log(Level.INFO, logger, false, "Available contexts: " + contextNames.toString());

    ArrayList<String> list = new ArrayList<String>(contextNames);
    TestUtils.log(Level.INFO, logger, false, "Available contexts in list " + list.toString());
               
    if(bChromeView) {
        //switch to Chrome View
        for(String chromContext : contextNames)
               if (chromContext.contains("chrome"))     //Android: select context by name
                        sWebViewContext = chromContext;
                if(sWebViewContext == "")
                        sWebViewContext = list.get(2);    //iOS: select context by index
    }
    else {
         //switch to App Web View
         for(String chromContext : contextNames)
                  if (chromContext.contains("ourappname"))
                          sWebViewContext = chromContext;
                   if(sWebViewContext == "")
                         sWebViewContext =  list.get(1);
     }
                
     TestUtils.log(Level.INFO, logger, false, "App WebView context: " + sWebViewContext);
     driver.context(sWebViewContext);
     return sWebViewContext;

}

The first method is executed when the user taps on the login button in our app. At that point the external website appears and the code switches to the Chromeview which is needed to interact with the external website. The second method is executed when the user completes the login on the external website (for the sake of shorteness I did not paste the code which enters credentials before tapping the button). At that point the callback is supposed to happen and our app is supposed to reappear. Therefore the code switches back to the Webview of our app.

The following log excerpts of the execution on an Android device show which contexts are available and which context is selected at the different steps.

At the beginning of the test execution we switch to the app webview which we use to automate our hybrid app:

15:46:50.276 [main] INFO  utils.TestUtils - Available contexts: [NATIVE_APP, WEBVIEW_com.package.name.of.our.app.main.test.debug]
15:46:50.277 [main] INFO  utils.TestUtils - found webview context: WEBVIEW_com.package.name.of.our.app.main.test.debug
15:46:50.277 [main] INFO  TC_0147b_InactiveForPrepaidClickNDrive - switching to context WEBVIEW_com.package.name.of.our.app.main.test.debug

Then, after the tap on the login button in our app, we switch to the Webview of Chrome which we use to automate the external website:

15:47:25.240 [main] INFO  utils.TestUtils - Available contexts: [NATIVE_APP, WEBVIEW_com.package.name.of.our.app.main.test.debug, WEBVIEW_chrome]
15:47:25.240 [main] INFO  utils.TestUtils - Available contexts in list [NATIVE_APP, WEBVIEW_com.package.name.of.our.app.main.test.debug, WEBVIEW_chrome]`
15:47:25.241 [main] INFO  utils.TestUtils - switching to context: WEBVIEW_chrome

Finally, after the tap on the login button on the external website, we switch back to the app webview:

15:47:38.112 [main] INFO  LoginPage - clicking element By.cssSelector: button.button.btn.btn-primary.btn-lg.login-button
15:47:48.874 [main] INFO  utils.TestUtils - Available contexts: [NATIVE_APP, WEBVIEW_com.package.name.of.our.app.main.test.debug, WEBVIEW_chrome]
15:47:48.875 [main] INFO  utils.TestUtils - Available contexts in list [NATIVE_APP, WEBVIEW_com.package.name.of.our.app.main.test.debug, WEBVIEW_chrome]
15:47:48.875 [main] INFO  utils.TestUtils -switching to context: WEBVIEW_com.package.name.of.our.app.main.test.debug

As written above, this all works fine in the iOS case but it does not for Android. In the Android case the callback does not reach the app, hence the external website remains on the screen and the test fails because it cannot interact with the app. We tried to wait for a very long time but it did not help, the webpage remains there. We also tried to close the external website after pressing the login button. While that brings the app back to the foreground, it does not solve the problem because the app remains in logged-out state and hence the test fails a bit farther when the test tries to execute actions in the app that only logged users can execute.

I am not sure that the problem is in the above code. The question is what is different in the Appium case with respect to the manual testing case so that the callback via custom URL does not work in combination with Appium (Android only). In both automatic and manual cases it is the same app and the same environment.

Does anybody have an idea?

2

Does the same test work being executed locally? It might be the cloud vendor applies some special security settings to their devices, thus preventing your scenario from working properly. Have you also tried to contact their customer support?

3

I just tried a local execution (with Appium desktop, that is, Appium server running on my PC in relaxed security and with a local Android device connected via USB). The result is exactly the same: The Appium test opens our app, pushes the login button in the app, enters the user credentials on the external website, presses the login button on that site but then nothing happens. Our app does not come to the foreground as expected. Instead, the external website remains in foreground. The login does not work, our app remains in logged-out state. So the very same behaviour as seen on the Bitbar devices. Again, for some reason which I don’t understand, the callback from external web application to our app (via custom URL scheme) does not work. So I think that the problem has nothing to do with the cloud vendor. Any other ideas what could cause the issue?

PS: Sorry for my late reply, I was off for a couple of days.

4

By the way, I tried to manually run the following command when the local device is connected:

adb shell am start -a android.intent.action.VIEW -d "nameofourapp://" package.name.of.our.app

and our app opened as expected. So the custom URL scheme works correctly if triggered manually. As stated above, the callback to the app with same custom URL scheme works also correctly in a manual test when triggered by the login webpage. Only when login page is used by an automatic Appium test, then the callback based on custom URL scheme does not work. I don’t know how and why Appium could interfere with it. At least I don’t see anything suspicious in the Appium server log. Before failing, the Appium test still sees all the contexts (APP_native, app webview, Chrome browser webview) and switches to the correct context (app webview): However, since the app is not in forground, the test fails when trying to accessing app elements. Does anybody have an idea what else I could try to make sure that the browser callback reaches the app?

5

Finally I managed to solve the problem.

My suspicions above that Appium itself is somehow (by design or by some setting) interfering with the custom URL callback done in the login process was completely wrong.

Instead, there was something wrong in our code. The problem was the explicit call of driver.get after the opening of the login webpage. That command was reloading the webpage, even though with the same URL as opened by our app. While that had no impact in the iOS case (Safari), it did disturb in the Android case (Chrome). I don’t know exactly why. I can just guess: looking at some videos of test executions, I got the impression that the call of driver.get on an already loaded page does not trigger a reload in Safari but it does on Chrome. And the reload must have had an impact on the logging process, most likely the identity server delivering that webpage does not like that it is called twice with identical parameters.

In any case, leaving away driver.get solves the issue. This thread can be closed.