Quantcast
Channel: Windows Desktop Development for Accessibility and Automation forum
Viewing all articles
Browse latest Browse all 585

Having trouble getting IUI_Automation Events to work

$
0
0

I followed Guy Barker's recommendation to build an interop wrapper around the UIAutomation.dll here and built my wrapper, added it to a new .NET Console project, and I'm just trying to get an event to fire on WindowOpen event:

using interop.UIAutomationCore;
using System;

namespace DMSMonitor
{

    class Program
    {
        static void Main(string[] args)
        {
            Scraper s = new Scraper();
            s.Init();
            Console.WriteLine("Press any key to quit...");
            while (true)
            {
                var info = Console.ReadKey(true);
                if (info.KeyChar == 's')
                {
                    // we cannot add an event handler inside another event handler https://stackoverflow.com/a/32786268/843567
                    // so we'll trigger subscribing to the PropertyChanged event handler by entering this key 's'
                    s.Subscribe();
                }
                else
                {
                    break;
                }
            }
        }
    }

    public class Scraper : IUIAutomationEventHandler, IUIAutomationPropertyChangedEventHandler, IUIAutomationFocusChangedEventHandler
    {

        private static string title = "MyAppTitle";
        private IUIAutomation uia = new CUIAutomation();
        private IUIAutomationElement root;
        private IUIAutomationElement dms;
        private IUIAutomationCacheRequest cacheReq;

        public void Init()
        {
            cacheReq = uia.CreateCacheRequest();
            root = uia.GetRootElementBuildCache(cacheReq);
            Console.WriteLine("added new window open event listener");
            uia.AddAutomationEventHandler(UIA_EventIds.UIA_Window_WindowOpenedEventId, root, TreeScope.TreeScope_Element, cacheReq, this);
            uia.AddAutomationEventHandler(UIA_EventIds.UIA_Window_WindowClosedEventId, root, TreeScope.TreeScope_Descendants, cacheReq, this);
            uia.AddAutomationEventHandler(UIA_EventIds.UIA_MenuOpenedEventId, root, TreeScope.TreeScope_Descendants, cacheReq, this);
            uia.AddFocusChangedEventHandler(cacheReq, this);
        }

        public void Subscribe()
        {
            if (dms != null)
            {
                SubscribePropertyChange(dms);
            }
        }

        [MTAThread]
        public void HandleAutomationEvent(IUIAutomationElement sender, int eventId)
        {
            Console.WriteLine("handle automation event, sender={0}, eventid={1}", sender.CurrentName, eventId);
            if (sender != null)
            {
                if (sender.CurrentName.Equals(title))
                {
                    dms = sender;
                    Console.WriteLine("new window opened with name {0}", sender.CurrentName);
                }
            }
        }

        private void SubscribePropertyChange(IUIAutomationElement element)
        {
            uia.AddPropertyChangedEventHandler(element, TreeScope.TreeScope_Element, null, this, new int[] { UIA_PropertyIds.UIA_NamePropertyId });
            Console.WriteLine("added subscription for NameProperty change on {0}", element.CurrentName);
        }

        [MTAThread]
        public void HandlePropertyChangedEvent(IUIAutomationElement sender, int propertyId, object newValue)
        {
            Console.WriteLine("property changed {0}, {1}, {2}", sender.CurrentName, propertyId, newValue);
        }

        [MTAThread]
        public void HandleFocusChangedEvent(IUIAutomationElement sender)
        {
            Console.WriteLine("focus changed {0}", sender.CurrentName);
        }
    }
}

When I run the app, it sits and waits (presumably) to fire the event handler when a new window is opened off the desktop / root element, but the event never occurs.  Here's what I see in the output, though...these native exceptions are thrown:

'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\clbcatq.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\UIAutomationCore.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\uxtheme.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\CoreMessaging.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\CoreUIComponents.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\SHCore.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ntmarta.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\WinTypes.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\oleacc.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sxs.dll'. Symbols loaded.
mincore\com\oleaut32\dispatch\ups.cpp(2125)\OLEAUT32.dll!779B69CF: (caller: 779B656E) ReturnHr(1) tid(276c) 8002801D Library not registered.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\OneCoreUAPCommonProxyStub.dll'. Symbols loaded.
'DMSMonitor.exe' (Win32): Loaded 'C:\Windows\SysWOW64\dwmapi.dll'. Symbols loaded.
The thread 0x285c has exited with code 0 (0x0).
The thread 0x421c has exited with code 0 (0x0).
The thread 0x1e98 has exited with code 0 (0x0).
The thread 0x3b70 has exited with code 0 (0x0).
mincore\com\oleaut32\dispatch\ups.cpp(2125)\OLEAUT32.dll!779B69CF: (caller: 779B656E) ReturnHr(2) tid(276c) 8002801D Library not registered.
mincore\com\oleaut32\dispatch\ups.cpp(2125)\OLEAUT32.dll!779B69CF: (caller: 779B656E) ReturnHr(3) tid(276c) 8002801D Library not registered.
Exception thrown at 0x7477D722 in DMSMonitor.exe: Microsoft C++ exception: wil::ResultException at memory location 0x06E8E3F0.
Exception thrown at 0x7477D722 in DMSMonitor.exe: Microsoft C++ exception: [rethrow] at memory location 0x00000000.
Exception thrown at 0x7477D722 in DMSMonitor.exe: Microsoft C++ exception: wil::ResultException at memory location 0x06E8DE68.
Exception thrown at 0x7477D722 in DMSMonitor.exe: Microsoft C++ exception: [rethrow] at memory location 0x00000000.
The thread 0x3ff0 has exited with code 0 (0x0).

When I launch any new application on the desktop, I would think the event handler fires, but it does not.  I'm looking for a specific app title in that code above, but nonetheless, I do a Console.WriteLine for any event and it never triggers.  I think the problem is related to the native exceptions it is throwing but I don't have any way to introspect those?

This is on Windows10, using VS 2017, just a simple C# Console App.

Any idea what I'm doing wrong here?

I did try using the .NET UI Automation API, and I was successful in getting the WindowOpen event to fire there, but I was having problems getting the PropertyChanged event to fire, so I started digging and that's when I found Guy Barker's posts, and decided to try the interop wrapper, but now I can't get the event to fire at all.



Viewing all articles
Browse latest Browse all 585

Trending Articles