Quantcast
Channel: Windows Desktop Development for Accessibility and Automation forum
Viewing all 585 articles
Browse latest View live
↧

Surrounding native window handle of UI Automation element

August 6, 2015, 3:59 am
$
0
0

Hello,

Is there any possibility to get the window containing an UI Automation element ?

ProcessId property gives me a process where the element comes from , but many applications have more than one native windows and I cannot match the element to window.

One possibility is to use TreeWalker and search for the parent which implements NativeWindowHandle property, but it does much more than I want. It iterates over each parent element what may result in slow run and increased CPU consumption.

I would like to see IUIAutomation.HandleFromElement to be almost the opposite of IUIAutomation.ElementFromHandle.

Thanks.

↧
Search

Windows 10 Start Menu: How to reach program utility links quickly?

August 11, 2015, 7:11 pm
$
0
0

In Windows 7, when a program installed, it installed a number of useful links to additional tools/information, too.

In Windows 10, only the program itself is visible in the start menu (see screenshot). But particularly for Microsoft Visual Studio I also need the tools that come along with Visual Studio.

In the right-pointing arrow next to the Start Menu entry, only the most recently used documents are available. - I would prefer the contents of the correspondingC:\ProgramData\Microsoft\Windows\Start Menu\Programs\ folder to also be available from there.

So my current question is: How can I quickly reach a program's upper mentioned Start Menu folder from within the Start Menu?

Image

Please also refer to this thread for further deails. They have pointed me here.

Still people out there alive using the keyboard?

Working with SQL Server/Office/Windows and their poor keyboard support they seem extinct...

↧
↧

windows push notification

August 13, 2015, 1:33 pm
$
0
0

I am having trouble with windows 10. How do I stop the windows push notification form continually interrupting me when I am doing something on the computer? How do I remove it? No matter what I am doing it breaks in and my computer starts flashing and will not stop until I shut the computer down. I also get a message that the Doby Audio Drive in not installed. I get a message that windows photo is not working. I think this means there is a problem with the installation of windows 10 but it has been removed from my computer and reinstalled already.

What can I try. I need my computer for work and this is no good.

Laurie

↧

How to get the control position based on the window

August 18, 2015, 2:59 am
$
0
0

How to get the control position based on the window like Left, Right, Bottom, TopLeft, TopRight etc.

↧

C# Z Order of All Open Windows

August 18, 2015, 7:24 am
$
0
0

I am interested in getting the Z order for every window currently open by the user. For example, if they had 4 windows on top of each other, I'd like to know (by window title - which I can already get), the order - i.e. which is deepest in the stack and which is at the front of the stack.

Does anyone have an idea as to the code for this? I've been looking at GetTopWindow() and GetNextWindow() but GetTopWindow() seems to return the top window within an application and GetNextWindow() doesn't seem to be referenced in user32.dll (got this from Pinvoke).

Any code ideas/hints would be gratefully received. I'm hoping there is just an integer value which is the Z index of the windows and from that, I can work out the order!

UPDATE: I have tried playing around with GetWindow() to no avail as when I use GW_HWNDNEXT it doesn't return the correct window handle. Basically all I want is a list of all windows currently open (both inside and outside my application) in the z order they appear on screen.

Cheers, Jonathan

↧
↧

How to duplicate display on two monitors using win32 APIs

August 18, 2015, 10:23 am
$
0
0

 I wanted to write one module in win32 which can detach all monitors and attach them back. It is working if no monitors are duplicated. But if two monitors are duplicated then two monitors are under same display eg Display3\Monitor0 and Display3\Monitor1. Detach using ChangeDisplaySettingsEx succeeds but reattach using same API and stored DEVMODE reattaches only one monitor to Display3 and the other monitor just remain disconnected. 

And ChangeDisplaySettingsEx failes if i give full monitor name(eg Display3\Monitor0). It just works for first argument being just DisplayName ( eg Display3).

So someone please guide me  how can i re-attach both monitors to Display3 as there were before detaching.

↧

How to do UI Automation for windows store app developed in WinJS?

August 18, 2015, 8:56 am
$
0
0

Hi,

I want to do UI automation of windows store app which is build using WinJS. Can you please help me how can I launch the Windows store app (WinJS) and do UI automation for my test scenarios?

Thanks in Advance!

Manoj D. 

↧

I want to know how to learn win32 api?

June 8, 2015, 6:25 pm
$
0
0


I want to know how to learn win32 use C++,,,,,,,,,,, very confused. I don't know where to learn,  do you have the chinese document?

↧
Search

Automate Login to a Third Party Application

May 6, 2015, 8:00 am
$
0
0

Hi Experts,

I am trying to develop a .Net Application for automation which is able to login to a third party application and perform a set of jobs which is currently performed manually

I am facing some Issues with the code i have written and need your Expert help.

Below is the Link to the code in OneDrive

https://onedrive.live.com/?cid=1f0b0affb1fd6044&id=1F0B0AFFB1FD6044%213197&ithint=file,zip&authkey=!AN6nBhsF9UX6nqg

1. When i run the Application, it breaks at the below line of code with Object reference Error

DicControl.Add("Password", sTextBoxControlList.Item(0).childHandle)
DicControl.Add("UserName", sTextBoxControlList.Item(1).childHandle)
DicControl.Add("Login", sLogin.Item(0).childHandle)

this is because sTexBoxControlList is empty.

The Weird part is when i debug the Code thru visual studio it passes thru the code with out any issue

2. I am trying to send some keys to the Handle to try to login but this is not working.

    ' Set Control to Password
    NativeWin32.SetFocus(DicControl("Password"))
    SendKeys.SendWait(Password)
    NativeWin32.SetFocus(DicControl("Login"))
    SendKeys.SendWait("{ENTER}")

Can you please help me out.

Thanks in advance.

Joji

↧
↧

How do I exposed to a UI Automation Client that an offset in my RichText editor contains a link?

April 7, 2015, 3:28 pm
$
0
0

How do I exposed to a UI Automation Client that an offset in my RichText editor contains a link?

I have a custom RichText editor in a C# WPF application.

What interface would expose such information? ITextRange? ITextRangeProvider?

When using a screen reader, I noticed when using WordPad if I up or down arrow to a line of text that contains "The search enginewww.bing.com" I will hear "The search engine linkwww.bing.com."  Notice the word "link" was added.

I would like to implement the same behavior in my custom RichText editor when navigating up and down by line.

When I right arrow through the text in WordPad, the screen reader does NOT speak "link" when the caret reaches the first "w" in "www.bing.com".

I would like to implement the speaking of "link" when the caret reaches the first character in the URL.

I assume the UIA client would call ITextRangeProvider.Move with TextUnit.Character when the user navigates with the left or right arrow keys.

Then through some other UI Automation means, the screen reader could determine there is a link at the TextRange.  How is this done?

I see the client can call ITextProvider.RangeFromChild which "Retrieves a text range enclosing a child element such as an image, hyperlink, or other embedded object."  Does the UIA client then iterate through the range and find the link? If so, is there an example on how to do so?

↧

What is the new API for reading text from GDI+?

January 29, 2015, 4:25 pm
$
0
0

Hi everyone,

I'm new to the forum, and have a question I have not been able to get an answer for: what is the new API replacing mirror drivers?  I even asked about it on Stack Overflow:

stackoverflow.com/questions/27851709/what-is-the-new-api-that-repalces-mirror-drivers-starting-in-windows-8

(links to my previous research on the subject there) and even they didn't seem to have any info (which is funny, because most of the stuff I ask on Stack Overflow usually gets at least one answer as well as several rude comments (lol)).

Basically, what I'm looking for is a way to get information from custom controls (and in particular the ones that use drawing APIs like GDI+) accessible.  But I'm talking about controls that have already been developed, and most of what I found here is referring to building accessibility into custom controls we design, probably using the UIA "Provider" API.  That is absolutely awesome (and I can't stress enough the importance "doing it right the first time") but in my current position I am developing accessibility solutions for existing applications.  I try to keep them as AT-neutral as possibleFor web-based apps this is usually done by manipulating the DOM in IE or writing extensions for other browsers, but desktop apps are a whole different ball game.

Anyway, before Windows 8 all the top AT products used mirror drivers (some for magnification, some for reading text etc.) but from what I've read they're deprecated in Windows 8+.  One of the links I included in my post on Stack Overflow talks about a new "special GDI accessibility API" that would replace them.  So if anyone knows what that API is I would really appreciate it.  Thanks. :)

↧

Speech Platform 11 Recognition in a Desktop App

February 7, 2015, 8:55 am
$
0
0

I'm doing some tests with speech recognition engine on my app, this is a sample grammar from my code:

******

GrammarBuilder _Start = new GrammarBuilder("start action");
Grammar _Actions = new Grammar(_Start);

******

This should recognize when i say "start action".

The problem i'm having is:

- It indeed recognize when i say "start action", but if i say "start start" it also recognize it the grammar.

And if i make the Grammar using Choices, its even worse.
*******

Choices _Action = new Choices();
_Action.Add(new string[] {"Recording", "Playback", "Transfer"});
GrammarBuilder gbAction = new GrammarBuilder();
gbAction.Append("Start");
gbAction.Append(_Action);
Grammar _Actions = new Grammar(gbAction);

******

If i say "start start".

I get "Start Recording" or "Start Playback", ..., recognized randomly.

This behavior is "normal"? or i'm missing somethng? some settings maybe?
I tried using Inputfrom wav file instead of DefaultInput, the results where the same.

Note: If i use a very long phrase like "start speech engine now", it doesn't recognize the "start start" as speech.
But then it becomes a very awkward voice interface for my app.

Using:

MS Speech Platform 11 SDK
MS Speech Runtime 11
OS : W7
Target CPU: X64

Thx in advance.

↧

Can not found html document in inspect.exe!

August 26, 2015, 10:44 pm
$
0
0

I have developed a program that uses UIAutomation.

Use the inspect.exe, and tried to verify the documents that are displayed in the IE, it will not be able to access the HTML document of active IE window.  But I can found html document in other PCs.

My Environment :

 Windows 7 64bit professional

 Internet Explorer 11


Could you help me or tell any idea?


↧
↧

TreeWalker.GetNextSibling method hangs thread

June 22, 2009, 2:30 pm
$
0
0
Due to the problems described in this thread: http://social.msdn.microsoft.com/Forums/en-US/windowsaccessibilityandautomation/thread/df34a3d0-b81e-4b86-a9ec-b8eb2ab77b7f
I've had to start relying on the TreeWalker class rather than FindFirst/FindAll to obtain AutomationElements for certain controls in our application.  However, I've found that when TreeWalker is instantiated using certain PropertyCondition objects, it seems to cause calls to GetNextSibling to lock up my test app's automation thread.

For example, say I want to find a control with a particular AutomationID as a child of a particular parent control.  I might use a method like this one:

public AutomationElement[] GetChildrenByAutomationId(AutomationElement parent, string autoId)
{
	PropertyCondition cond = new PropertyCondition(AutomationElement.AutomationIdProperty, autoId);
	TreeWalker tw = new TreeWalker(cond);
	AutomationElement[] children = new AutomationElement[] { };
	AutomationElement child = tw.GetFirstChild(parent);
	while (child != null)
	{
		children = children.Concat(new AutomationElement[] { child }).ToArray();
		child = tw.GetNextSibling(child);
	}

	return children;
}
In this case, the initial call to tw.GetFirstChild returns successfully, however during the first pass through the while loop, execution hangs at the call to tw.GetNextSibling.

I sometimes (but not always) see one or more debug messages pop up in the Visual Studio output window that say:

A first chance exception of type 'System.Runtime.InteropServices.COMException' occurred in UIAutomationClientsideProviders.dll

but other than thay I get no indication of what's gone wrong, and the call to GetNextSibling never returns.


Jesse Kindwall
↧

UI Automation events stop being received after a while and then restart after some time

September 2, 2015, 1:22 am
$
0
0

We are using Microsoft's UIAutomation framework to develop a client that monitors events of a specific application and responds to them in different ways. We've started with the managed version of the framework, but due to delay issues, moved to the native version wrapped in UIACOMWrapper (can't add link). After more issues with performance inside our (massive) WPF application, we decided to move it to a separate terminal application (transfer the events to our WPF app through UDP) which seemed to fix all the performance issues. The only problem is that it seems that every several minutes, the events for TabSelection, StructureChanged, WindowOpened and WindowClosed stop being captured for a few minutes. Surprisingly PropertyChanged events are still received and handled while this happens. The application continues to perform as usual with windows and dialogs being opened but no events captured other than name and value changes.

In order to find out if this is a bug caused by our code or the wrapper, I tried usingMicrosoft's own AccEvent utility to capture the same events from the monitored application and gotsimilar results (some events stop being captured for periods of time). This leads me to conclude that there is a fundamental bug in the framework somewhere. I can't post the code of the monitored application as it is proprietary and confidential as well, I can say that it is a WinForms application that hosts WPF windows and also quite massive.

Has anyone seen this sort of behavior while working with the UI Automation framework? Thank you for your time.

EDIT: I forgot to mention that I strongly suspect that the issue is that one of the UI-Automation event handler threads gets stuck somehow. The reason I believe this, is that when the problem occurred in my monitor, I started an instance of AccEvent and it received all the missing events that my monitor was not getting. This means that the events are being fired but not passed to my monitor.
↧
Search

Application Info for Modern/Universal "Apps"

September 2, 2015, 10:48 am
$
0
0

The following sample has returned the name of the process that is associated with the active window, but does not work with the newer modern/universal apps because it returns a helper processWWAHost.exe on Windows 8 and ApplicationFrameHost.exe on Windows 10.

HWND active_window = GetForegroundWindow();
GetWindowThreadProcessId(active_window,&active_process_id );
HANDLE active_process = OpenProcess(
    PROCESS_QUERY_INFORMATION | PROCESS_VM_READ,
    FALSE, active_process_id );
GetProcessImageFileName( active_process,
    image_name, 512 );

With Windows 10 the ApplicationFrameHost.exe is the process that creates the window handles and is what gets returned by GetWindowThreadProcessId(), is there another Win32 API that can be used to get the active process of universal app that is active?

Also tried using GetApplicationUserModelId() and GetPackageFullName() with no success as they return APPMODEL_ERROR_NO_APPLICATION and APPMODEL_ERROR_NO_PACKAGE respectively because the active_process handle is just the helper process and not the process of the active application.

Any other APIs to use to get the process name of a Modern/Universal application given the hwnd of the window, or otherwise figure out the process name of the universal app is active.

Thanks in advance!

↧

windows narrator and visula ui automation verify issue

July 26, 2015, 7:02 am
$
0
0

When i look for sub windows of a particular app using Visual UIAutomation Verify it doesn't show the sub windows but when i use windows narrator to read the app's main window-name then only Visual UIAutomation Verify shows the child windows. Why is this happening?

↧
↧

Using UIAutomation via COM (Interop.UIAutomationClient.dll) causes Windows Explorer to crash

July 28, 2015, 8:35 am
$
0
0

We have recently moved from using the managed UIAutomation libraries in favor of the UIAComWrapper project which wraps the functionality in the Interop.AutomationClient.dll so that we could maintain our namespaces across our project that uses UIAutomation extensively. Since then, we have had reports of random windows explorer crashes and were able to narrow down the cause and reliably reproduce the issue. This issue does not occur while using the managed UIAutomationClient, however since that is no longer being developed we would much rather take advantage of the native API as it has shown us better results, aside from this crash. See the sample code at the bottom of this post as well as a link to the UIAComWrapper project source code on git hub.

Registering for StructureChangedEvents on the RootElement, with the TreeScope of Children will cause Windows Explorer to crash intermittently. I have been able to reliably cause a crash by either navigating between network drives in Windows Explorer, or more reliably by right clicking the desktop -> Personalize -> Desktop Background and clicking the drop down "Picture Position". The crash occurs at the same location every time no matter how it is reproduced in Windows Explorer, in DUI70.dll, with the same offset every time, with an exception code C0000005. When attached to Explorer.exe in visual studio, I was able to download the public symbols for DUI70.dll and when the exception occured the function that DUI70.dll was currently executing was"RaiseStructureChangeEvent". I was only able to view function headers and the rest of disassembly. 

Is anyone else able to reproduce this and have some insight as to what may be causing it? 

UIAComWrapperhttps://github.com/JakeGinnivan/UIAComWrapper

 


↧

Win 10. FindFirst returns null. TreeWalker throws exception

September 17, 2015, 9:37 am
$
0
0
In about 30% cases search for my top element returns null. Occures randomly only on Win10. Even within same test if on earlier steps it found top form it might not find it next time. 

AutomationElement shell = AutomationElement.RootElement.FindFirst(TreeScope.Children,
new PropertyCondition(AutomationElement.NameProperty, _shellElementName));
 if (shell == null)
{
shell = AutomationElement.RootElement.FindFirst(TreeScope.Children,
        new PropertyCondition(AutomationElement.AutomationIdProperty, _shellElementId));
}

_shellElementName & _shellElementId are consts and top form never changes. I also triedTreeScope.Descendants -- made no difference except being slower. Subsequent

TreeWalker walker = TreeWalker.ControlViewWalker;
AutomationElement childElement = walker.GetFirstChild(AutomationElement.RootElement);

throws exception:

System.Windows.Automation.ElementNotAvailableException: Element not available ---> System.ArgumentException: Value does not fall within the expected range.

System.ArgumentException: Value does not fall within the expected range.
   at UIAutomationClient.IUIAutomationTreeWalker.GetFirstChildElementBuildCache(IUIAutomationElement element, IUIAutomationCacheRequest cacheRequest)
   at System.Windows.Automation.TreeWalker.GetFirstChild(AutomationElement element, CacheRequest request)
System.Windows.Automation.ElementNotAvailableException: Element not available ---> System.ArgumentException: Value does not fall within the expected range.
   at UIAutomationClient.IUIAutomationTreeWalker.GetFirstChildElementBuildCache(IUIAutomationElement element, IUIAutomationCacheRequest cacheRequest)
   at System.Windows.Automation.TreeWalker.GetFirstChild(AutomationElement element, CacheRequest request)
   --- End of inner exception stack trace ---
   at System.Windows.Automation.TreeWalker.GetFirstChild(AutomationElement element, CacheRequest request)
   at Test.CodedUI.Desktop.Controls.ControlNavigator.GetChildrenNames(AutomationElement automationElement)
 
Any suggestions?



↧

Bug in UI Automation with Metro apps when text element scrolled beyond first line

September 17, 2015, 10:45 am
$
0
0

Hi, I am trying to extract the text under the mouse using the following code

	private static async Task<string> getText(double x, double y)        {            try            {                var location = new System.Windows.Point(x, y);                AutomationElement element = AutomationElement.FromPoint(location);                               object patternObj;                if (element.TryGetCurrentPattern(TextPattern.Pattern, out patternObj))                {                    var textPattern = (TextPattern)patternObj;                    var range = textPattern.RangeFromPoint(location);                    range.ExpandToEnclosingUnit(TextUnit.Word);                                        var text = range.GetText(-1).Trim();                    range.ExpandToEnclosingUnit(TextUnit.Line);                    text += " [" + range.GetText(-1).Trim() + "]";                    return text;                }                else if (element.TryGetCurrentPattern(ValuePattern.Pattern, out patternObj))                {                    var valuePattern = (ValuePattern)patternObj;                    return valuePattern.Current.Value;                }                else                {                    var text = "";                    var patterns = element.GetSupportedPatterns();                    foreach (var pattern in patterns)                    {                        text += pattern.Id + " " + pattern.ProgrammaticName;                        text += Environment.NewLine;                    }                    return text;                }            }            catch (Exception ex)            {                return ex.Message;            }        }

this works normally (using the Money Metro app to test). However, if the text box is scrolled beyond the first line, the UI Automation begins to pull the wrong text because it thinks the top of the window is the top of the text. For example for the image below, the mouse is on "increasing" and the top of the text (starting with "A majority...") is visible.

before scrolling mouse is on

If we scroll past the first line, and place the mouse again on "increasing", this time it gets confused and is pulling from the line above. As we scroll more and more the displacement becomes larger and larger. In the image below, it picks up "with" from the line above.



Any ideas for workaround also much appreciated.

↧
Viewing all 585 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>