Pinch and zoom on an image in IOS 7 simulator

Support
1

Hi guys,

Any ideas on how to pinch and zoom on a particular image on the app, I’m using ruby and automating on the IOS 7.1 simulator. I need to pinch and zoom on the image and not the whole page. Swipe seems to have broken. I’ve tried $driver.pinch 75 and zoom but that seems to hit the corners of the app which causes it to navigate elsewhere. I’ve also tried

action1 = Appium::TouchAction.new.swipe(start_x: 124, start_y: 330, end_x: 68, end_y: 397, duration: 1)
action2 = Appium::TouchAction.new.swipe(start_x: 208, start_y: 316, end_x: 276, end_y: 253, duration: 1)
multi_action=Appium::MultiTouch.new
multi_action.add action1
multi_action.add action2
multi_action.perform

Doesn’t work! if I tweak the coordinate values I get
Selenium::WebDriver::Error::JavascriptError: point is not within the bounds of the screen

Am I in the right direction? Please help!

Thanks
Aish

2

It looks like you’re heading in the right direction :smile:

If the points are out of bounds, try tweaking them until they are in bounds, the slowly expand them?

There should be a method for getting the current bounds of the screen.

Instead of the swipe action, try building the actions more manually. You just need to combine the events:

  • touchDown
  • moveTo
  • touchUp
4

Hi @jonahss , I tried that but the actions don’t seem to be reflected. I added the muti touch

[{:action=>:press, :options=>{:element=>“2”, :x=>150, :y=>300}},
{:action=>:moveTo, :options=>{:element=>“2”, :x=>45, :y=>372}},
{:action=>:release, :options=>{:element=>“2”}}],
[{:action=>:press, :options=>{:element=>“2”, :x=>190, :y=>300}},
{:action=>:moveTo, :options=>{:element=>“2”, :x=>283, :y=>250}},
{:action=>:release, :options=>{:element=>“2”}}]]

This is my muti touch object. The appium response is :

info: --> POST /wd/hub/session/22433c80-586d-43e8-9507-49b6738145d4/touch/multi/perform {“actions”:[[{“action”:“press”,“options”:{“element”:“2”,“x”:150,“y”:300}},{“action”:“moveTo”,“options”:{“element”:“2”,“x”:45,“y”:372}},{“action”:“release”,“options”:{“element”:“2”}}],[{“action”:“press”,“options”:{“element”:“2”,“x”:190,“y”:300}},{“action”:“moveTo”,“options”:{“element”:“2”,“x”:283,“y”:250}},{“action”:“release”,“options”:{“element”:“2”}}]]}

debug: Appium request initiated at /wd/hub/session/22433c80-586d-43e8-9507-49b6738145d4/touch/multi/perform

debug: Request received with params: {“actions”:[[{“action”:“press”,“options”:{“element”:“2”,“x”:150,“y”:300}},{“action”:“moveTo”,“options”:{“element”:“2”,“x”:45,“y”:372}},{“action”:“release”,“options”:{“element”:“2”}}],[{“action”:“press”,“options”:{“element”:“2”,“x”:190,“y”:300}},{“action”:“moveTo”,“options”:{“element”:“2”,“x”:283,“y”:250}},{“action”:“release”,“options”:{“element”:“2”}}]]}
debug: Pushing command to appium work queue: “au.getElement(‘2’).rect()”
debug: Sending command to instruments: au.getElement(‘2’).rect()
debug: Sending command to instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Got new command 25 from instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluating au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluation finished

(This is where I think my actions going wrong)
debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Result is not protocol compliant, wrapping

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Running system command #26: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}…

debug: Socket data received (80 bytes)

debug: Socket data being routed.
debug: Got result from instruments: {“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}
debug: Pushing command to appium work queue: “au.getElement(‘2’).rect()”
debug: Sending command to instruments: au.getElement(‘2’).rect()
debug: Sending command to instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Got new command 26 from instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluating au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluation finished

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Result is not protocol compliant, wrapping

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Running system command #27: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}…

debug: Socket data received (80 bytes)

debug: Socket data being routed.
debug: Got result from instruments: {“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}
debug: Pushing command to appium work queue: “au.getElement(‘2’).rect()”
debug: Sending command to instruments: au.getElement(‘2’).rect()
debug: Sending command to instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Got new command 27 from instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluating au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluation finished

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Result is not protocol compliant, wrapping

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Running system command #28: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}…

debug: Socket data received (80 bytes)

debug: Socket data being routed.
debug: Got result from instruments: {“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}
debug: Pushing command to appium work queue: “au.getElement(‘2’).rect()”
debug: Sending command to instruments: au.getElement(‘2’).rect()
debug: Sending command to instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Got new command 28 from instruments: au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluating au.getElement(‘2’).rect()

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluation finished

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Result is not protocol compliant, wrapping

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Running system command #29: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}…

debug: Socket data received (80 bytes)

debug: Socket data being routed.
debug: Got result from instruments: {“status”:0,“value”:{“origin”:{“x”:0,“y”:0},“size”:{“width”:320,“height”:64}}}
debug: Pushing command to appium work queue: “target.touch([{“time”:0.2,“touch”:[{“x”:150,“y”:300},{“x”:190,“y”:300}]},{“time”:0.4,“touch”:[{“x”:45,“y”:372},{“x”:283,“y”:250}]}])”
debug: Sending command to instruments: target.touch([{“time”:0.2,“touch”:[{“x”:150,“y”:300},{“x”:190,“y”:300}]},{“time”:0.4,“touch”:[{“x”:45,“y”:372},{“x”:283,“y”:250}]}])
debug: Sending command to instruments: target.touch([{“time”:0.2,“touch”:[{“x”:150,“y”:300},{“x”:190,“y”:300}]},{“time”:0.4,“touch”:[{“x”:45,“y”:372},{“x”:283,“y”:250}]}])

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Got new command 29 from instruments: target.touch([{“time”:0.2,“touch”:[{“x”:150,“y”:300},{“x”:190,“y”:300}]},{“time”:0.4,“touch”:[{“x”:45,“y”:372},{“x”:283,“y”:250}]}])

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluating target.touch([{“time”:0.2,“touch”:[{“x”:150,“y”:300},{“x”:190,“y”:300}]},{“time”:0.4,“touch”:[{“x”:45,“y”:372},{“x”:283,“y”:250}]}])

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: target.touch(__NSCFArray)

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: evaluation finished

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Command executed without response

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Result is not protocol compliant, wrapping

debug: [INST] 2014-09-06 15:12:57 +0000 Debug: Running system command #30: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:""}…

debug: Socket data received (25 bytes)

info: <-- POST /wd/hub/session/22433c80-586d-43e8-9507-49b6738145d4/touch/multi/perform 200 845.113 ms - 74 {“status”:0,“value”:"",“sessionId”:“22433c80-586d-43e8-9507-49b6738145d4”}

debug: Socket data being routed.
debug: Got result from instruments: {“status”:0,“value”:""}
debug: Responding to client with success: {“status”:0,“value”:"",“sessionId”:“22433c80-586d-43e8-9507-49b6738145d4”}

Could you please help?

Thanks
Aish

5

Yeah, looks like a problem with the way the action is being marshaled and sent to the server. Which client are you using?

6

I’m using the ruby client. Server version 1.2.2

7

maybe @bootstraponline can see an issue?

8

I suggest opening an issue on the ruby client repository.

9

I am trying to do gestures using the Java bindings on a native iOS table view. I am trying something like this, but I can’t compile - the MultiTouch class is not found:

    TouchAction ta1 = new TouchAction(driver);
    TouchAction ta3 = new TouchAction(driver);
    ta1.press(element1);
    ta3.moveTo(element3);
    MultiTouch mt = new MultiTouch();
    //add the actions and do a perform...
10

Okay, I see that the class I need to use is MultiTouchAction

11

I don’t find touchUp or touchDown methods.

BTW, are the coordinate args to the press method relative to an element, or to the screen?

12

Even something as simple as this (ios simulator, native app) throws an InvocationTargetException. Where am I going wrong? I’m trying to work around the absence of a scrollToBottom method.

    driver = new AppiumDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
    MobileElement element1 = (MobileElement) sodaIosService.findElement(By.xpath("//UIATableView/UIATableCell[1][@name='myPointsHistoryCell']"));
    TouchAction ta1 = new TouchAction(driver);
    TouchAction ta2 = new TouchAction(driver);
    ta1.press(element1);
    ta2.release();
    MultiTouchAction ma = new MultiTouchAction(driver);
    ma.add(ta1).add(ta2);
    ma.perform();
13

Maybe the element can’t be found? Make sure it’s there.
Your actions are incomplete. You want a single action that presses then releases.

ta1.press(element1).release().perform()

The above is a single action chain. Remember that each TouchAction object is like a single finger. So the finger presses, then releases. MultiTouchAction would be used if you wanted two fingers to tap the screen at the same time.

ta1.press(element1).release();
ta2.press(element2).release();
ma.add(ta1).add(ta2).perform();
14

Thanks jonah. Will those actions cause the screen to scroll? I’m trying to figure out a way to scroll to the bottom of the screen.

15

If you want to scroll, then simulate the same action that your fingers would do.

You will have to experiement, but it would be something like this:

ta1.press(element1).moveTo([x and y coordinates which are maybe 100px above the element]).release(); ta2.press(element2).moveTo([x and y coordinates which are maybe 100px above the element]).release(); ma.add(ta1).add(ta2).perform();

And that should be two fingers “scrolling” upwards. Now repeat it until you hit the bottom…

16

Hi Jonahss

I am trying to zoom/shrink image on Real device , ipad app, C# bindings.
However i am getting invalid javascript error everytime
below is the code:

MultiAction ac = new MultiAction(driver);
TouchAction ta1 = new TouchAction(driver);
TouchAction ta2 = new TouchAction(driver);
ta1.Press(ele,373,289).MoveTo(300,336).Release();
ta2.Press(ele,450,276).MoveTo(526, 240).Release();
ac.Add(ta1).Add(ta2).Perform();

Please suggest, I dont think problem is with coordinates… whatever coordinates i provide., it throws invalid javascript.

17

Its a typo or you really have MoveTo instead of moveTo and Release() instead of release() and so on?

I never worked with appium/java, but java is case sensitive

18

hey i am using c# client… and these are correct function names… there is some other issue…
it says invalid javascript

19

Hey Richa,

ta1.Press(373,289).MoveTo(300,336).Release();
ta2.Press(450,276).MoveTo(526, 240).Release();

If you are giving coordinates their is no need to write the element name.

20

c#, appium-1.3.1, ios
Can anybody tell how to draw a circle using touchaction

Thanks

21

Just before using zoom and pinch ,
just give a single touch to the element that you need to zoom in or pinch
Here is what i have done and works perfectly…

List imageParent = (List) getDriver().findElements(By.className(“android.support.v4.view.ViewPager”));
WebElement imageChild = imageParent.get(0).findElement(By.className(“android.widget.ImageView”));
firstScreenShot=image.screenshot(imageChild);
int leftX = imageChild.getLocation().getX();
int rightX = leftX + imageChild.getSize().getWidth();
int upperY = imageChild.getLocation().getY();
int lowerY = upperY + imageChild.getSize().getHeight();
int middleY = (upperY + lowerY) / 2;
int middleX = (leftX + rightX) / 2;
TouchAction action0=new TouchAction(getDriver()).press(middleX,middleY).release();
action0.perform();
getDriver().zoom(middleX,middleY);

1 Like