To run an Android app in a virtual display and retrieve its window UI tree on AOSP (Android Open Source Project) using Java/Android APIs, you’ll need to combine several Android system-level features. Here’s a high-level approach and the key APIs/libraries you’ll use:

1. Create a Virtual Display

A virtual display allows you to render an app’s UI into a virtual screen, which you can then capture or inspect.

Key Classes/APIs

• VirtualDisplay (via WindowManager or DisplayManager)
• SurfaceControl (for capturing the display)
• MediaProjection (for screen mirroring/capture)

Steps:

1. Request MediaProjection permission (if capturing the screen):
   java

   Copy

   // Request user permission for screen capture
   Intent intent = mediaProjectionManager.createScreenCaptureIntent();
   startActivityForResult(intent, REQUEST_CODE);

2. Create a VirtualDisplay:
   java

   Copy

   VirtualDisplay virtualDisplay = displayManager.createVirtualDisplay(
       "VirtualDisplay",
       width, height, dpi,
       surface,
       DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
       null, null);

   • Use DisplayManager to create the virtual display.
   • Attach a Surface to it.

2. Retrieve the Window UI Tree

To inspect the app’s window hierarchy (e.g., for UI testing or accessibility), use:

Key Classes/APIs

• AccessibilityService (for UI tree inspection)
• WindowManager (for window hierarchy)
• UiAutomation (for dumping the UI hierarchy)

Steps:

1. Implement an AccessibilityService:

   java

   Copy

   public class MyAccessibilityService extends AccessibilityService {
       @Override
       public void onAccessibilityEvent(AccessibilityEvent event) {
           // Handle accessibility events (e.g., window state changes)
       }
       @Override
       public void onInterrupt() {}
   }

   • Declare it in your AndroidManifest.xml.

2. Dump the UI Hierarchy:

   java

   Copy

   UiAutomation uiAutomation = getUiAutomation();
   AccessibilityNodeInfo root = uiAutomation.getRootInActiveWindow();
   // Traverse the tree or dump it to XML/JSON
   Log.d("UI_TREE", root.toString());

3. Use adb shell dumpsys window (for debugging):

   bash

   Copy

   adb shell dumpsys window windows

3. Capture the Virtual Display

If you want to capture the virtual display (e.g., for screenshots or video):

• Use SurfaceControl to create a screenshot:
  java

  Copy

  SurfaceControl screenshot = SurfaceControl.screenshot(
      virtualDisplay.getDisplay().getDisplayId());

• Or use MediaProjection to record the virtual display.

4. Putting It All Together

Here’s a rough outline for a system app:

Manifest Permissions

<uses-permission android:name="android.permission.MEDIA_PROJECTION" />
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<uses-permission android:name="android.permission.ACCESSIBILITY_SERVICE" />

Example Code

// 1. Create VirtualDisplay
DisplayManager displayManager = (DisplayManager) getSystemService(DISPLAY_SERVICE);
VirtualDisplay virtualDisplay = displayManager.createVirtualDisplay(
    "VirtualDisplay",
    1080, 1920, 320,
    surface,
    DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
    null, null);

// 2. Start AccessibilityService
Intent intent = new Intent(this, MyAccessibilityService.class);
startService(intent);

// 3. Dump UI Tree
UiAutomation uiAutomation = getUiAutomation();
AccessibilityNodeInfo root = uiAutomation.getRootInActiveWindow();

5. Notes

• Requires system privileges: On AOSP, many of these APIs are restricted to system apps or require root.
• SELinux policies: Ensure your app has the right SELinux context to access these APIs.
• Performance: Virtual displays and UI tree dumps can be resource-intensive.
• Alternatives: For UI testing, consider UiAutomator or Espresso.

References

• Android VirtualDisplay Documentation
• AccessibilityService Guide
• MediaProjection API

Would you like a more detailed code example for a specific part (e.g., virtual display setup or UI tree dumping)? Or are you targeting a specific AOSP version?