How use semantic labels in appium flutter

Emmanuel_Anokye · May 19, 2025, 12:45pm

I want to trigger interactions on my widgets through their semantic labels but for some reason appium isn’t able to identify the semantic labels on the widgets. The semantic labels are visible in the FlutterWidget inspector though so i can’t figure out why its not able to find it. Any help??

wreed · May 19, 2025, 3:29pm

What driver? There are currently 2 Flutter drivers available:

And there are use cases for both Xcuitest and Uiautomator2:

Since Flutter 3.19, Flutter apps can expose identifier for SemanticsProperties as resource-id in Android and accessibilityIdentifier in iOS.

Whichever one you are using, you might want to try one of the others as suits your needs best.

Emmanuel_Anokye · May 19, 2025, 5:05pm

I’m using the appium flutter integration driver. Trying to use it for end to end testing of the flutter app

wreed · May 19, 2025, 8:34pm

I don’t know Flutter all that well, but could you be looking at this issue:

github.com/flutter/flutter

Not able to find element by Semantics in Release build

opened 01:54PM - 03 Jul 24 UTC

saikrishna321

framework a: accessibility f: integration_test a: release has reproducible steps P2 team-framework triaged-framework found in release: 3.22 found in release: 3.23

### Steps to reproduce Unable to find element by Semantics ### Expected resul…ts Should be able to find element bySemanticsLabel ### Actual results No element is found ### Code sample <details open><summary>Code sample</summary> FileName - test.dart ```dart import 'package:appium_testing_app/main.dart'; import 'package:flutter/cupertino.dart'; import 'package:flutter_test/flutter_test.dart'; import 'package:integration_test/integration_test.dart'; void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); group('end-to-end test', () { testWidgets('tap on the floating action button, verify counter', (tester) async { // Load app widget. await tester.pumpWidget(const MyApp()); // Finds the floating action button to tap on. final fab = find.bySemanticsLabel('username_text_field'); print(fab.evaluate()); List<DiagnosticsNode> nodes = tester .widget(fab) .toDiagnosticsNode() .getProperties(); // Emulate a tap on the floating action button. await tester.tap(fab); for (DiagnosticsNode node in nodes) { print("Flutter - ${node.name} -> ${node.value}"); } // Trigger a frame. await tester.pumpAndSettle(); }); }); } ``` App Code ``` Semantics( label: "username_text_field", key: const Key("username_text_field"), // textField: true, explicitChildNodes: true, container: true, child: TextField( decoration: const InputDecoration.collapsed(hintText: "Username"), controller: userNameController, ), ), ``` </details> Command Used to build the IPA ``` flutter build ipa --release --export-options-plist=/Documents/ExportOptions.plist /Downloads/git/appium-flutter-server/demo-app/integration_test/test.dart ``` ### Screenshots or Video <details open> <summary>Screenshots / Video demonstration</summary> [Upload media here] </details> ### Logs <details open><summary>Logs</summary> ```console {"msg":"Jul 3 19:22:35 God- kernel(AppleMobileFileIntegrity)[0] \u003cNotice\u003e: AMFI: constraint violation /private/var/containers/Bundle/Application/394C6BFC-9825-4665-A685-48F161BF8357/Runner.app/Frameworks/webview_flutter_wkwebview.framework/webview_flutter_wkwebview has entitlements but is not a main binary"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: 00:00 +0: end-to-end test tap on the floating action button, verify counter"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: Found 0 widgets with element matching predicate: []"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: Bad state: No element"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #0 Iterable.single (dart:core/iterable.dart:674)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #1 WidgetController.widget (package:flutter_test/src/controller.dart:801)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #2 main.\u003canonymous closure\u003e.\u003canonymous closure\u003e (file:///Users/saikrishna/Downloads/git/appium-flutter-server/demo-app/integration_test/test.dart:17)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: \u003casynchronous suspension\u003e"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #3 testWidgets.\u003canonymous closure\u003e.\u003canonymous closure\u003e (package:flutter_test/src/widget_tester.dart:183)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: \u003casynchronous suspension\u003e"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #4 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: \u003casynchronous suspension\u003e"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: #5 TestWidgetsFlutterBinding._createTestCompletionHandler.\u003canonymous closure\u003e (package:flutter_test/src/binding.dart:809)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: \u003casynchronous suspension\u003e"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: 00:00 +0: end-to-end test tap on the floating action button, verify counter [E]"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: Test failed. See exception logs above.\n The test description was: tap on the floating action button, verify counter"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter:"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: 00:00 +0 -1: (tearDownAll)"} {"msg":"Jul 3 19:22:36 God- Runner(Flutter)[66332] \u003cNotice\u003e: flutter: 00:00 +1 -1: Some tests failed."} ``` </details> ### Flutter Doctor output <details open><summary>Doctor output</summary> ```console [Paste your output here] ``` </details>

Looks like it can happen in debug builds too. I don’t see this as being currently fixed, unfortunately. Got it from the link you cited:

Testing on real iOS devices for semanticsLabel may not work due to an issue raised with Flutter. For updates and more information, please refer to GitHub issue #151238.