Version 1.3.1: pinch seems to innacurately translate on server - RESOLVED

In experimenting with the pinch calls via io.appium / java-client / version 2.0.0,
and running against Appium server 1.3.1, driving an iOS application,
we realized that pinch(WebElement), pinch(x,y) and MultiTouchCommand all ultimately resolve to the creation of a MultiTouchCommand.

After no success with either pinch call, we tried coding the MultiTouch as follows.
(Calculate midX and midY and say pinch from above/below that point ± 100)


        Dimension dimension = driver.manage().window().getSize();
        int midY = dimension.getHeight() / 2;
        int midX = dimension.getWidth() / 2;
        int yOffset = 100;
        TouchAction thumb = new TouchAction(driver).press(midX, midY - yOffset).moveTo(midX, midY).release();
        TouchAction forefinger = new TouchAction(driver).press(midX, midY + yOffset).moveTo(midX, midY).release();
        MultiTouchAction multiTouch = new MultiTouchAction(driver);
        multiTouch.add(thumb).add(forefinger);
        multiTouch.perform();

Given that the screen size was 1024 x 768, the mids were 512 and 384, respectively.

When you call multiTouch.perform, it calls driver.performMultiTouchCommand, which posts the command to the server.

You see the post to the server as (formatted for readability):


POST /wd/hub/session/bf031f56-c6e8-478b-8b53-f32d27db8b21/touch/multi/perform 
{"actions":
     [
         [
          {"action":"press","options":{"y":412,"x":384}},
          {"action":"moveTo","options":{"y":512,"x":384}},
          {"action":"release","options":{}}
         ],
         [
          {"action":"press","options":{"y":612,"x":384}},
          {"action":"moveTo","options":{"y":512,"x":384}},
          {"action":"release","options":{}}
         ]
     ]
}

Looks good!
But then when the server turns that into a command, somehow it comes up with (seen in Appium Server Log):


Pushing command to appium work queue: 
"target.touch(
    [
        {
            \"time\":0.2,
            \"touch\":
            [
                {\"x\":384,\"y\":412},
                {\"x\":384,\"y\":612}
            ]
        },
        {
            \"time\":0.4,
            \"touch\":
            [
               {\"x\":768,\"y\":924},
               {\"x\":768,\"y\":1124}
            ]
        }
    ]
)"

So… a couple things:

  • the starting positions for the pinch look correct.... both at midX and +-100 from midY
  • the ending positions seem to be weird, x's are double the starting position's x, y's are starting position's y's + 512.
  • the y=1124 in the ending position is causing the next lines in the appium server log, which are:

info: [debug] Sending command to instruments: target.touch([{"time":0.2,"touch":[{"x":384,"y":412},{"x":384,"y":612}]},{"time":0.4,"touch":[{"x":768,"y":924},{"x":768,"y":1124}]}])

info: [debug] [INST] 2014-11-05 21:30:08 +0000 Debug: Got new command 22 from instruments: target.touch([{"time":0.2,"touch":[{"x":384,"y":412},{"x":384,"y":612}]},{"time":0.4,"touch":[{"x":768,"y":924},{"x":768,"y":1124}]}])

info: [debug] [INST] 2014-11-05 21:30:08 +0000 Debug: evaluating target.touch([{"time":0.2,"touch":[{"x":384,"y":412},{"x":384,"y":612}]},{"time":0.4,"touch":[{"x":768,"y":924},{"x":768,"y":1124}]}])

info: [debug] [INST] 2014-11-05 21:30:08 +0000 Debug: target.touch(__NSCFArray)

info: [debug] [INST] 2014-11-05 21:30:08 +0000 Debug: point is not within the bounds of the screen

Are we misunderstanding the moveTo, or the construction of the MutliTouch?
We realize we could tweak the numbers and make it work mathematically.
Just wondering where the misconception is.

Any advice is appreciated.

Looked at the Javadoc for the moveTo. It says:


TouchAction io.appium.java_client.TouchAction.moveTo(int x, int y)
Move current touch to an absolute position on the screen
Parameters:
   x x coordinate
   y y coordinate
Returns:
  this TouchAction, for chaining

Presuming that the misconception might in fact be that it’s relative, rather than absolute, position that moveTo wants, we changed the code to the following and it works.


        final int offset = 100;
        
        WebElement mainWindow = driver.findElement(By.xpath("...
        Dimension mainWindowSize = mainWindow.getSize();
        int midY = mainWindowSize.getHeight() / 2;
        int midX = mainWindowSize.getWidth() / 2;
               
        TouchAction finger1 = new TouchAction(driver).press(midX + offset, midY).moveTo((offset * -1), 0).release();
        TouchAction finger2 = new TouchAction(driver).press(midX - offset, midY).moveTo(offset, 0).release();

        new MultiTouchAction(driver).add(finger1).add(finger2).perform();

So you basically have to say “move from” origin ± offset in whatever direction (in this case horizontally), and “move to” [negate the offset (give relative position)] and specify 0 (zero relative change) in the vertical.

Hope this helps someone else.

1 Like

@pihlenfeldt Thanks for the solution. I saw this behavior as well and thought that the moveTo is relative.

@jonahss this causes bugs in the pinch and zoom functions (in java_client), from AppiumDriver:

public void pinch(int x, int y) {
    MultiTouchAction multiTouch = new MultiTouchAction(this);

    int scrHeight = manage().window().getSize().getHeight();
    int yOffset = 100;

    if (y - 100 < 0) {
        yOffset = y;
    } else if (y + 100 > scrHeight) {
        yOffset = scrHeight - y;
    }

    TouchAction action0 = new TouchAction(this).press(x, y - yOffset).moveTo(x, y).release();
    TouchAction action1 = new TouchAction(this).press(x, y + yOffset).moveTo(x, y).release();

    multiTouch.add(action0).add(action1);

    multiTouch.perform();
}