Is ClickAndHold(WebElement canvas) supported by Appium?

selinawoo · November 6, 2015, 5:29am

I used Selenium supported action “clickAndHold” to draw a line on our mobile web canvas using iPhone simulator Safari, but it generated “WebDriverException: Not yet implemented …” in the error log.
The same selenium code was working fine when I ran it on desktop app without Appium.
Is clickAndHold supported by Appium?

Here is the whole error message:
org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 5 milliseconds

Here is my java code running on Eclipse:
try {
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
WebElement inputCanvas = driver.findElement(By.id(“listenerLayer”));

    	Actions actionBuilder = new Actions(driver);
    	
    	driver.findElement(By.cssSelector(".toolbar-item.color.red")).click();

    	actionBuilder.clickAndHold(inputCanvas).
		 moveByOffset(-100, -200).moveByOffset(-200,-100).moveByOffset(100,200).
		 moveByOffset(200,100).release().perform();
    }
    catch (Exception e) {
        	System.err.println(e.toString());
    }

Please advise.

Thanks a lot!

afwang · November 6, 2015, 8:38pm

You could give the TouchAction API a try. This API supports more general tapping, pressing, and dragging operations than the usual operations directly available from AppiumDriver.

selinawoo · November 6, 2015, 10:40pm

On the TouchAction page, I saw the constructors. Do you have any examples that I can learn how to use it?

Thanks!

selinawoo · November 7, 2015, 12:01am

I changed the code with the example of the reference page, but it is complaining about the driver input. It required "Cast argument from driver to Mobile Driver’.

import io.appium.java_client.TouchAction;
import io.appium.java_client.MobileDriver;

WebDriver driver = null;
TouchAction action = new TouchAction(driver);

What code do I need to add so that it can take the argument?

Thanks,
Selina.

selinawoo · November 7, 2015, 1:23am

I used TouchAction and it is also not working. It generated the same error:
org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 7 milliseconds
Build info: version: ‘2.48.2’…

Here is the code that I used with TouchAction. These are the only lines that changed.

           public IOSDriver driver = null;
           driver = new IOSDriver(new URL("http://localhost:4723/wd/hub"), cap);

    	TouchAction action = new TouchAction(driver);
    	
    	action.longPress(inputCanvas).moveTo(200, 200).moveTo(250, 250)
    	.release().perform();

Please let me know whether Appium supports canvas actions which I am mainly automating.

Thanks,
Selina.

afwang · November 7, 2015, 5:25am

Let me list out what I think you’re trying to do:

Have a “finger” come down and press on the center of a UI element with a reference named “inputCanvas”.
Drag the finger along a straight line to a point that is 200 pixels to the right and 200 pixels down.
From the finger’s new position, drag the finger along a straight line to a point 250 pixels to the right and 250 pixels down.
Bring the finger up.

If this is accurate, you should try starting off with the press method rather than the longPress method (I’m pretty sure that longPress method isn’t completely implemented on Appium - I hope someone corrects me on that if I’m wrong).

There is also 1 thing I would like to point out. I remember writing the current documentation for the moveTo() method, but it’s important to note that the behavior of this method changed somewhere between Appium versions 1.3 and 1.4. When the method was first introduced, it would move the “finger” relatively. The last time I checked on a newer version of Appium, moveTo would move the “finger” absolutely (I’ll probably have to open a pull request to fix this after checking myself). You’ll probably have to do a bit of arithmetic in your code to get the numbers you want. To do this, get the dimension and position of your inputCanvas element so you can calculate the position of its center. Then add 200 to the x and y coordinates for the first moveTo call. Then add 250 to the new x and y coordinates to get the parameters for the second moveTo call.

selinawoo · November 9, 2015, 9:37pm

In fact, I just want to automate some drawing (e.g. a line) on the canvas.
Right now, appium won’t let me draw anything.

I changed the longpress to press, and it still generated the same error.

    	TouchAction action = new TouchAction(driver);
    	
    	action.press(inputCanvas).moveTo(200, 200).release().perform();

Here is the error:

org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 6 milliseconds
Build info: version: ‘2.48.2’, revision: ‘41bccdd’, time: ‘2015-10-09 19:55:52’
System info: host: ‘SHWOO-M-G0GP’, ip: ‘10.154.228.60’, os.name: ‘Mac OS X’, os.arch: ‘x86_64’, os.version: ‘10.9.5’, java.version: ‘1.8.0_60’
Driver info: io.appium.java_client.ios.IOSDriver
Capabilities [{networkConnectionEnabled=false, desired={}, safari=true, warnings={}, webStorageEnabled=false, locationContextEnabled=false, browserName=iOS, takesScreenshot=true, javascriptEnabled=true, databaseEnabled=false, platform=MAC}]
Session ID: 1d61521a-33b5-4d2c-b52a-e69ba94dd86a

Please let me know whether Appium is able to automate drawing on canvas.

Thanks,
Selina.

selinawoo · November 9, 2015, 11:37pm

I am using the Actions instead of TouchAction. At least, I saw a dot on the canvas.
With TouchAction, I always saw “Not yet implemented” error.

Here is my code working partially with the original x,y coord with a dot.
However, I cannot draw a line from the existing x,y coord to the orangePen element.
Since the clickAndHold is not implemented by Appium,
what should the syntax be in order to draw a line with Actions.
Thanks.

Actions actionBuilder = new Actions(driver);
Action drawOnCanvas= actionBuilder.moveToElement(inputCanvas, 20, 169)
.click()
.moveToElement(orangePen)
.click()
.build();
drawOnCanvas.perform();

afwang · November 10, 2015, 3:59am

What does Appium’s server logs say? You can find these logs in the Appium GUI window, or if you’re running Appium on the command line, you can find them in the terminal window where Appium is ran. Generally, the information contained in the Exception thrown from the Java test is not detailed enough to diagnose a problem properly.

selinawoo · November 10, 2015, 5:53am

Here is the code:
Actions actionBuilder = new Actions(driver);
actionBuilder.moveToElement(inputCanvas, 20, 169).click().perform();
System.out.println(“Before clickAndHold!”);
actionBuilder.clickAndHold(inputCanvas).moveByOffset(100, 100)
.release().perform();
System.out.println(“After clickAndHold!”);

Here is the error message:
Before clickAndHold!
org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 5 milliseconds
Build info: version: ‘2.48.2’, revision: ‘41bccdd’, time: ‘2015-10-09 19:55:52’
System info: host: ‘xxx’, ip: ‘y.y.y.y’, os.name: ‘Mac OS X’, os.arch: ‘x86_64’, os.version: ‘10.9.5’, java.version: ‘1.8.0_60’
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Capabilities [{networkConnectionEnabled=false, desired={}, safari=true, warnings={}, webStorageEnabled=false, locationContextEnabled=false, browserName=iOS, takesScreenshot=true, javascriptEnabled=true, databaseEnabled=false, platform=MAC}]
Session ID: 8cb93336-a59a-49c9-87b7-cb254d287518

selinawoo · November 10, 2015, 9:12pm

Here is the Appium log on the GUI:

info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/element/5006/click 200 24.729 ms - 76 {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: --> POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto {“yoffset”:169,“xoffset”:20,“element”:“5004”}
info: [debug] [REMOTE] Executing ‘get_top_left_coordinates’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] [REMOTE] Executing ‘move_mouse’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto 200 36.998 ms - 76 {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: --> POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/click {“button”:0}
info: [debug] Pushing command to appium work queue: “au.getElementsByType(‘webview’)”
info: [debug] Sending command to instruments: au.getElementsByType(‘webview’)

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Got new command 9 from instruments: au.getElementsByType(‘webview’)

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluating au.getElementsByType(‘webview’)

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluation finished

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: responding with:
info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Running system command #10: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/submodules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:[{“ELEMENT”:“4”}]}…

info: [debug] Socket data received (40 bytes)

info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {“status”:0,“value”:[{“ELEMENT”:“4”}]}
info: [debug] Pushing command to appium work queue: “au.getElement(‘4’).rect()”
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] Sending command to instruments: au.getElement(‘4’).rect()
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Got new command 10 from instruments: au.getElement(‘4’).rect()

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluating au.getElement(‘4’).rect()
info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluation finished

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: responding with:
info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Running system command #11: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/submodules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:{“origin”:{“x”:0,“y”:64},“size”:{“width”:320,“height”:460}}}…
info: [debug] Socket data received (82 bytes)
info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {“status”:0,“value”:{“origin”:{“x”:0,“y”:64},“size”:{“width”:320,“height”:460}}}
info: [debug] Converted web coords {“x”:20,“y”:169}into real coords {“x”:20,“y”:233}
info: [debug] Pushing command to appium work queue: “au.complexTap({“x”:20,“y”:233,“tapCount”:1,“duration”:0.3,“touchCount”:1})”
info: [debug] Sending command to instruments: au.complexTap({“x”:20,“y”:233,“tapCount”:1,“duration”:0.3,“touchCount”:1})

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Got new command 11 from instruments: au.complexTap({“x”:20,“y”:233,“tapCount”:1,“duration”:0.3,“touchCount”:1})

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluating au.complexTap({“x”:20,“y”:233,“tapCount”:1,“duration”:0.3,“touchCount”:1})

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: target.tapWithOptions({x:“20”, y:“233”}, {touchCount:“1”, tapCount:“1”, duration:“0.3”})

info: [debug] Socket data received (25 bytes)

info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {“status”:0,“value”:""}
info: [debug] Responding to client with success: {“status”:0,“value”:"",“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/click 200 697.850 ms - 74 {“status”:0,“value”:"",“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}

info: --> POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto {“element”:“5004”}
info: [debug] [REMOTE] Executing ‘get_top_left_coordinates’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] [REMOTE] Executing ‘move_mouse’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger

info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto 200 24.460 ms - 76 {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: --> POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto {“element”:“5004”}
info: [debug] [REMOTE] Executing ‘get_top_left_coordinates’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: evaluation finished

info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: responding with:
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] [REMOTE] Executing ‘move_mouse’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [INST] 2015-11-10 21:01:52 +0000 Debug: Running system command #12: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/submodules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{“status”:0,“value”:""}…
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/moveto 200 23.845 ms - 76 {“status”:0,“value”:null,“sessionId”:“328866e1-0ae0-44cf-abf7-5b2c3de73c6a”}
info: --> POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/buttondown {}
info: [debug] Responding to client that a method is not implemented
info: <-- POST /wd/hub/session/328866e1-0ae0-44cf-abf7-5b2c3de73c6a/buttondown 501 1.214 ms - 158

info: [IOS_SYSLOG_ROW ] Nov 10 13:01:55 — last message repeated 13 times —

info: [IOS_SYSLOG_ROW ] Nov 10 13:01:55 Selina
lsuseractivityd[16867]: [SFActivityAdvertiser] ERROR: no XPC proxy, queuing advertisement <55391abd 70f90f01 e8>

info: [IOS_SYSLOG_ROW ] Nov 10 13:01:57 Selina mstreamd[16798]: (Note ) PS: Media stream daemon stopping.

selinawoo · November 12, 2015, 6:12pm

Any update on this? Can Appium support clickAndHold?

Thanks,
Selina.

selinawoo · November 12, 2015, 10:25pm

In case the previous log is too large, here is the line that is causing problem. It generated error code 501.

Thanks for investigating!

Selina.

ARP · January 3, 2017, 1:30pm

Is this issue still open?

selinawoo · January 3, 2017, 8:18pm

I am not using Appium to automate anymore, so you can close this if you want.

Thanks,
Selina.