Drag & Drop Hello World Example

Baubak_G · September 22, 2014, 4:34pm

Hi There,

I am trying to make a simple Drag & Drop gesture using TouchAction in Appium 1.2.2 .I am testing this on the site
Touch Punch Unfortunately I cannot get it to work. I am not sure I have understood this, or if tere are any known limitations. I have written a simple code

@Test
public void testHW_IOS_IPad_Appium_DragAndDrop() throws MalformedURLException, InterruptedException {
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
	capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "7.1");
	capabilities.setCapability(MobileCapabilityType.BROWSER_NAME, "Safari");
	capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "My IPAD");

	AppiumDriver driver = new AppiumDriver(new URL("http://<my Appium Server>:4723/wd/hub"),capabilities);
	driver.get("**http://touchpunch.furf.com/content.php?/droppable/default-functionality**");

	WebElement l_draggable = driver.findElement(By.id("draggable"));
	WebElement l_droppable = driver.findElement(By.id("droppable"));
	
	TouchAction action = new TouchAction(driver);
	action.press(l_draggable).perform();
	
	action = new TouchAction(driver);
	action.press(l_draggable).moveTo(l_droppable).release().perform();
	
	driver.quit();
}

Unfortunately nothing really happens. Here is a log exerpt:

Debugger web socket received data :  {"result":{"result":{"type":"string","value":"{\"status\":0,\"value\":{\"ELEMENT\":\":wdc:1411402623981\"}}"},"wasThrown":false},"id":3}
info: [debug] Responding to client with success: {"status":0,"value":  {"ELEMENT":"5001"},"sessionId":"337fc0a5-7403-43f5-8bfd-1a8425dc2a46"}
info: <-- POST /wd/hub/session/337fc0a5-7403-43f5-8bfd-1a8425dc2a46/element 200 50.546 ms - 90  {"status":0,"value":{"ELEMENT":"5001"},"sessionId":"337fc0a5-7403-43f5-8bfd-1a8425dc2a46"}
info: --> POST /wd/hub/session/337fc0a5-7403-43f5-8bfd-1a8425dc2a46/touch/perform {"actions":[{"action":"press","options":{"element":"5000"}}]}
info: [debug] Pushing command to appium work queue: "au.getElement('5000').rect()"
info: [debug] Sending command to instruments: au.getElement('5000').rect()
info: [debug] Didn't get a new command in 60 secs, shutting down...
info: Shutting down appium session
info: [debug] Stopping safariLauncher

Are there any limitations I am unaware about? Any halp is much appreciated.

Best Regards,

Baubak

Baubak_G · September 22, 2014, 5:08pm

I have also tried performing a swipe instead, using the positions, but it still doesn’t work:

WebElement l_draggable = driver.findElement(By.id("draggable"));
WebElement l_droppable = driver.findElement(By.id("droppable"));
    
driver.swipe(l_draggable.getLocation().getX(), l_draggable.getLocation().getY(), l_droppable.getLocation().getX(), l_droppable.getLocation().getY(), 1000);

Here is what I get in the log:

 info: --> POST /wd/hub/session/475a9673-effa-4e53-9f81-5a0ac5b91324/touch/perform {"actions":  [{"action":"press","options":{"y":18,"x":8}},{"action":"wait","options":{"ms":1000}},{"action":"moveTo","options": {"y":18,"x":146}},{"action":"release","options":{}}]}
info: [debug] Pushing command to appium work queue: "au.dragApp(8,18,146,18,1)"
info: [debug] Sending command to instruments: au.dragApp(8,18,146,18,1)
info: [debug] Didn't get a new command in 60 secs, shutting down...
info: Shutting down appium session
info: [debug] Stopping safariLauncher
info: [debug] Stopping iOS log capture

Baubak_G · September 24, 2014, 8:22am

Hello,

Can anyone provide me with some guidance on this topic?

Best regards,

Baubak

Baubak_G · September 26, 2014, 12:17pm

For your information, I can reproduce this on both Real Devices and the simulator:
SDK 7.1
Appium 1.2.2

The difference when I use the simulator is that Appium crashes:
info: [debug] Responding to client with success: {“status”:0,“value”:"",“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: <-- POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/url 200 2011.528 ms - 74 {“status”:0,“value”:"",“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: --> POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element {“using”:“id”,“value”:“draggable”}
info: [debug] [REMOTE] Executing ‘find_element’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:{“ELEMENT”:“5000”},“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: <-- POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element 200 18.229 ms - 90 {“status”:0,“value”:{“ELEMENT”:“5000”},“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: --> POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element {“using”:“id”,“value”:“droppable”}
info: [debug] [REMOTE] Executing ‘find_element’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:{“ELEMENT”:“5001”},“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: <-- POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element 200 18.013 ms - 90 {“status”:0,“value”:{“ELEMENT”:“5001”},“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: --> GET /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element/5001/text {}
info: [debug] [REMOTE] Executing ‘get_text’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Responding to client with success: {“status”:0,“value”:“Drop here”,“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: <-- GET /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/element/5001/text 200 23.882 ms - 83 {“status”:0,“value”:“Drop here”,“sessionId”:“f9778768-6166-484c-a096-ce4c0d559640”}
info: --> POST /wd/hub/session/f9778768-6166-484c-a096-ce4c0d559640/touch/perform {“actions”:[{“action”:“press”,“options”:{“element”:“5000”}},{“action”:“moveTo”,“options”:{“element”:“5001”,“y”:15,“x”:15}}]}
info: [debug] Pushing command to appium work queue: “au.getElement(‘5000’).rect()”
info: [debug] Sending command to instruments: au.getElement(‘5000’).rect()
info: [debug] Sending command to instruments: au.getElement(‘5000’).rect()
info: [debug] [INST] 2014-09-26 12:09:26 +0000 Debug: Got new command 5 from instruments: au.getElement(‘5000’).rect()
info: [debug] [INST] 2014-09-26 12:09:26 +0000 Debug: evaluating au.getElement(‘5000’).rect()
info: [debug] Socket data received (91 bytes)
info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {“status”:17,“value”:"‘null’ is not an object (evaluating ‘au.getElement(‘5000’).rect’)"}
error: uncaughtException: Cannot read property ‘x’ of undefined date=Fri Sep 26 2014 14:09:26 GMT+0200 (CEST), pid=13634, uid=502, gid=20, cwd=/usr/local/lib/node_modules/appium, execPath=/usr/local/Cellar/node/0.10.31/bin/node, version=v0.10.31, argv=[node, /usr/local/bin/appium, --tmp, /Users/rdneolane/Documents/BGA/Appium/tempDir/], rss=119107584, heapTotal=63371520, heapUsed=27315352, loadavg=[4.16357421875, 3.98046875, 3.1103515625], uptime=106156, trace=[column=36, file=/usr/local/lib/node_modules/appium/lib/devices/ios/ios-controller.js, function=, line=2128, method=null, native=false, column=43, file=/usr/local/lib/node_modules/appium/node_modules/async/lib/async.js, function=next, line=801, method=null, native=false, column=16, file=/usr/local/lib/node_modules/appium/node_modules/async/lib/async.js, function=null, line=32, method=null, native=false, column=9, file=/usr/local/lib/node_modules/appium/lib/devices/common.js, function=exports.respond, line=28, method=respond, native=false, column=18, file=/usr/local/lib/node_modules/appium/lib/devices/ios/ios.js, function=, line=1368, method=null, native=false, column=20, file=/usr/local/lib/node_modules/appium/node_modules/appium-uiauto/lib/command-proxy.js, function=getResultAndSendNext, line=146, method=null, native=false, column=7, file=/usr/local/lib/node_modules/appium/node_modules/appium-uiauto/lib/command-proxy.js, function=, line=88, method=null, native=false, column=20, file=events.js, function=Socket.emit, line=117, method=emit, native=false, column=16, file=_stream_readable.js, function=null, line=943, method=null, native=false, column=13, file=node.js, function=process._tickDomainCallback, line=463, method=_tickDomainCallback, native=false], stack=[TypeError: Cannot read property ‘x’ of undefined, at null. (/usr/local/lib/node_modules/appium/lib/devices/ios/ios-controller.js:2128:36), at next (/usr/local/lib/node_modules/appium/node_modules/async/lib/async.js:801:43), at /usr/local/lib/node_modules/appium/node_modules/async/lib/async.js:32:16, at exports.respond (/usr/local/lib/node_modules/appium/lib/devices/common.js:28:9), at null. (/usr/local/lib/node_modules/appium/lib/devices/ios/ios.js:1368:18), at getResultAndSendNext (/usr/local/lib/node_modules/appium/node_modules/appium-uiauto/lib/command-proxy.js:146:20), at Socket. (/usr/local/lib/node_modules/appium/node_modules/appium-uiauto/lib/command-proxy.js:88:7), at Socket.emit (events.js:117:20), at _stream_readable.js:943:16, at process._tickDomainCallback (node.js:463:13)]

jonahss · September 26, 2014, 7:01pm

Maybe it’s because you are using elements inside a webview?
Try doing the touchActions with just coordinates, and try switching to the NativeContext. Might work

Can definitely simulate gestures on WebViews, since I did it in this example

Baubak_G · September 28, 2014, 9:57am

Thanks I’ll certainly give it a try!

Baubak_G · October 2, 2014, 6:55am

Hi @jonahss

Looking at the example I see you specify :

var caps = {
platformName: ‘iOS’,
platformVersion: ‘7.1’,
deviceName: ‘iPad’,
app: '/Users/jonahss/Workspace/appium/sample-code/apps/WebViewApp/build/Release-iphonesimulator/WebViewApp.app’
}

Do I need to specify the app WebViewApp as well? What is its purpose? Isn’t setting Safari enough?

BEst regards,

Baubak

Baubak_G · October 2, 2014, 1:03pm

Hi again

Thanks @jonahss your example helped me to debug the functionality. I have now made it work but I have made a few discoveries.

When you use getLocation(WebElement), the result is in reference to the web page itself.
When you perform TouchActions (like press and move) they are in reference to the screen. (quite unexpected, but unfortuantely it is WAP (work as planned)).
BUG : In the documentation for the moveTo method (Both 1.7 & 2.0), we state Move current touch to an absolute position on the screen. However this is incorrect! We actually move the move to a point in reference to the original point.
BUG ? : These actions do not work on a real device

Example (using code from the Jonah’s example using Java). In this example we draw a horizontal line:

URL url = new URL("http://10.131.151.227:4723/wd/hub");
AppiumDriver driver = new AppiumDriver(url, capabilities);
driver.get("http://lukasolson.github.io/multi-touch-draw/");
action.press(100,100).moveTo(163, 0).release().perform();

The line below creates a vertical line that is 100px long:

 action.press(100,100).moveTo(0, 100).release().perform();

jonahss · October 2, 2014, 6:22pm

Can you submit a pull request to the documentation to fix it?

Can you post the bug report on the appium github repo?

Thanks

Baubak_G · October 7, 2014, 7:38am

Hi

Just for information, do I submit this to the JavaClient or directly on the Appium server branch?

The documentation I used was the javadocs for the TouchAction.moveTo() method

jonahss · October 7, 2014, 5:23pm

If you want to add to the Javadocs for the java-client, you can submit to the java-client repo.
If you want to add some guiding words to the docs found on appium.io you can submit changes to the files found under /docs in the appium repo.

richa_gupta · December 30, 2014, 9:07am

c#, appium-1.3.1, ios
Can anybody tell how to draw a circle using touchaction

Thanks

Baubak_G · February 4, 2015, 9:28am

Hi,

I had to put Appium aside for a short while, and have started anew.

It seems however that ever since 1.2.4 much has changed. I have installed Appium 1.3.4 (IOS 8.1), and now my drag&drop tests don’t work anymore.

I saw that the rules for the clients have changed, but (AppiumDriver being abstract) in many ways:

You can no longer cast RemoteWebDriver to AppiumDriver
The drag&drop hello world doesn’t work anymore. I now get the “Not yet implemented.” message.

@jonahss does your example still work on 1.3.4?

Best regards,

Baubak

Baubak_G · February 10, 2015, 4:42pm

Hi could anyone shed light on this please?

@jonahss The tests I performed on your example now give me the error:

org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)

Any help on this would be greatly appreciated.

Cheers,

Baubak

Appium_Master · February 11, 2015, 7:27am

@Baubak_G :- For drag and drop you can watch my video

Baubak_G · February 11, 2015, 8:21am

Hi @Appium_Master, as a matter of fact I have already. (And by the way, thanks for your videos). The difference between our cases is that I am testing a web page using iPad Safari

I still get the message :
org.openqa.selenium.WebDriverException: Not yet implemented. Please help us: http://appium.io/get-involved.html (WARNING: The server did not provide any stacktrace information)

PS: In my case I am using a real web page.

I adapted my code to look more like yours, but the problem remans the same.

Here is the code:

        DesiredCapabilities capabilities = new DesiredCapabilities();
  
        capabilities.setPlatform(Platform.valueOf("MAC"));
        capabilities.setBrowserName("iPad");
        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
        capabilities
                .setCapability(MobileCapabilityType.PLATFORM_VERSION, "8.1");
        capabilities.setCapability(MobileCapabilityType.BROWSER_NAME, "");

        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME,
                "iPad Retina");

        capabilities.setJavascriptEnabled(true); 
        AppiumDriver driver = new IOSDriver(<Appium server Address>,
                    fetchCapabilities());
        driver.get("http://touchpunch.furf.com/content.php?/droppable/default-functionality");
        driver.manage().timeouts().implicitlyWait(20L, TimeUnit.SECONDS);
    
        WebElement l_dragable = driver.findElement(By.id("draggable"));
        
        WebElement l_droppable = driver.findElement(By.id("droppable"));
        
        Assert.assertEquals(l_droppable.getText(), "Drop here");
        
        TouchAction dragNDrop = new TouchAction((MobileDriver)driver).press(l_dragable).moveTo(l_droppable).release();
        dragNDrop.perform();
        
        l_droppable = driver.findElement(By.id("droppable"));
        Assert.assertEquals(l_droppable.getText(), "Dropped!");
        driver.close();
        driver.quit();

Any help is most welcome.

PS: This worked in 1.2.4 (if you gave coordinates).

Baubak_G · February 11, 2015, 3:34pm

Hi I have analyzed this a bit more, and it seems that although TouchActions work on the WebViewApp they absolutely do not work on Safari iPad (on the silulator).

I would really be happy to be proven wrong.

In the meantime I have created the following issue for this:

garyst1981 · May 18, 2015, 12:51pm

I’m having same problem with Ruby + Chrome + Android. When using

min_slider = current_driver.find_element(:css, ".slider-handle.min-slider-handle.round")
Appium::TouchAction.new.tap(:element => min_slider).perform

It shows me the Not Implemented yet error

Not yet implemented. Please help us: http://appium.io/get-involved.html (Selenium::WebDriver::Error::UnknownError)

server version 1.3.6
appium_lib 4.1.0
android lollipop
Chrome 42

Some logs from the server

info: [debug] Proxying command to 127.0.0.1:9515
info: [debug] Making http request with opts: {"url":"http://127.0.0.1:9515/wd/hub/session/6339e5ae06a7d8d380d2e95ea14d18f5/element/0.5839314325712621-10/displayed","method":"GET"}
info: [debug] Proxied response received with status 200: "{\"sessionId\":\"6339e5ae06a7d8d380d2e95ea14d18f5\",\"status\":0,\"value\":true}"
info: <-- GET /wd/hub/session/6339e5ae06a7d8d380d2e95ea14d18f5/element/0.5839314325712621-10/displayed 200 83.015 ms - 72
info: --> POST /wd/hub/session/6339e5ae06a7d8d380d2e95ea14d18f5/touch/perform [{"action":"tap","options":{"element":"0.5839314325712621-11"}}]
info: [debug] Responding to client that a method is not implemented

kapil16 · July 6, 2017, 4:47pm

Hi @Baubak_G

I am getting an below issue , can you please help. Thanks in Advance:

ENVIORNMENT:
Appium version : 1.6.5 this i am using from command line.
Last Appium version that did not exhibit the issue (if applicable): v1.1.0-beta.4 (this is UI version) {This is with UI}
Desktop OS/version used to run Appium: Window 7 proffesional / 64 bit
Node.js version (unless using Appium.app|exe): Node.js 6.11.0 (x64) and npm
Mobile platform/version under test: 4.2.1
Real device or emulator/simulator: Real Device
Appium CLI or Appium.app|exe: Appium CLI

Issue: `java.lang.ClassCastException: org.openqa.selenium.remote.RemoteWebDriver cannot be cast to io.appium.java_client.MobileDriver
at com.qa.DragAndDrop.DragandDrop.getDesireCapabilities(DragandDrop.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

`

My Code is :

package com.qa.DragAndDrop;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import io.appium.java_client.MobileDriver;
import io.appium.java_client.TouchAction;
import io.appium.java_client.remote.MobileCapabilityType;

public class DragandDrop
{
WebDriver driver;
String device = “Android”;
String devicename = “0123456789ABCDEF”;
String platformversion = “4.2.1”;
String platformname = “Android”;

@Test
public void getDesireCapabilities() throws MalformedURLException, InterruptedException
{
	
	
	DesiredCapabilities cap = new DesiredCapabilities();
	cap.setCapability("device", device);
	cap.setCapability(MobileCapabilityType.DEVICE_NAME, devicename);
	cap.setCapability(MobileCapabilityType.PLATFORM_NAME, platformname);
	cap.setCapability(MobileCapabilityType.PLATFORM_VERSION, platformversion);
	cap.setCapability("browserName", "");
	cap.setCapability("appPackage", "com.mobeta.android.demodslv");
	cap.setCapability("appActivity", "com.mobeta.android.demodslv.Launcher");
	cap.setCapability(MobileCapabilityType.NEW_COMMAND_TIMEOUT, "100");
	driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"),cap);
	driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	driver.findElement(By.xpath("//android.widget.TextView[@text='Basic usage playground']")).click();
	
	WebElement source = driver.findElement(By.xpath("//android.widget.TextView[@text='Chick Corea']"));
	WebElement destination = driver.findElement(By.xpath("//android.widget.TextView[@text='Wayne Shorter']"));
	
	TouchAction action = new TouchAction((MobileDriver)driver);
	System.out.println("Dragging item");
	action.longPress(source).moveTo(destination).release().perform();
	System.out.println("Dragged Succssfully");
	//driver.quit();
	
}

}

Free_Automation · July 9, 2017, 9:43am

Try this https://youtu.be/zqjEXembEos