Hi, I am trying to extract the text under the mouse using the following code
private static async Task<string> getText(double x, double y) { try { var location = new System.Windows.Point(x, y); AutomationElement element = AutomationElement.FromPoint(location); object patternObj; if (element.TryGetCurrentPattern(TextPattern.Pattern, out patternObj)) { var textPattern = (TextPattern)patternObj; var range = textPattern.RangeFromPoint(location); range.ExpandToEnclosingUnit(TextUnit.Word); var text = range.GetText(-1).Trim(); range.ExpandToEnclosingUnit(TextUnit.Line); text += " [" + range.GetText(-1).Trim() + "]"; return text; } else if (element.TryGetCurrentPattern(ValuePattern.Pattern, out patternObj)) { var valuePattern = (ValuePattern)patternObj; return valuePattern.Current.Value; } else { var text = ""; var patterns = element.GetSupportedPatterns(); foreach (var pattern in patterns) { text += pattern.Id + " " + pattern.ProgrammaticName; text += Environment.NewLine; } return text; } } catch (Exception ex) { return ex.Message; } }this works normally (using the Money Metro app to test). However, if the text box is scrolled beyond the first line, the UI Automation begins to pull the wrong text because it thinks the top of the window is the top of the text. For example for the image below, the mouse is on "increasing" and the top of the text (starting with "A majority...") is visible.
If we scroll past the first line, and place the mouse again on "increasing", this time it gets confused and is pulling from the line above. As we scroll more and more the displacement becomes larger and larger. In the image below, it picks up "with" from the line above.