How to get selected text and its coordinates from any system wide application using Accessibility API?

In my previous post, I had discussed about accessing the text value from any system wide application by using Accessibility API. Now going further,  you may want to access only the selected text or the position of selected text, we will discuss about it in this article.

As mentioned in my previous post, for using Accessibility API you must have enabled access for assistive devices in Universal Access preference pane in System Preferences. I suggest you to go through this post first to know more about accessibility API before moving on with this article.

Now, Suppose you want to create a application where a global shortcut key could access active application’s (For example: TextEdit) selected text value and the position coordinates of selected text. This can be done using Accessibility API in the following way:

1. Create a system wide element to retrieve the current focused element of any application
AXUIElementRef systemWideElement = AXUIElementCreateSystemWide();
2. Retrieve the current focused element using kAXFocusedUIElementAttribute.
AXUIElementRef focussedElement = NULL;
AXError error = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute, (CFTypeRef *)&focussedElement);
if (error != kAXErrorSuccess)
{
NSLog(@”Error while retrieving focused element”);
}
else…..

3. Now, if the focused element is successfully retrieved, we will access the selected text. The attribute kAXSelectedTextAttribute is used to copy the focused element’s selected text’s value to our CFTypeRef object.

AXValueRef selectedText = NULL;
AXError selectedTextError = AXUIElementCopyAttributeValue(focussedElement, kAXSelectedTextAttribute, (CFTypeRef *)&selectedText);          //Copy selected text attribute from focussedElement to selectedText
if (selectedTextError == kAXErrorSuccess)
{
NSString* selectedTextString = (NSString *)(selectedText);
NSLog(@”%@”, selectedTextString);            // Selected text value retrieved
}
else
{
NSLog(@”Error while retrieving selected text”);
}

4. Now, we will see how to get the range and rect bounds of selected text. Note that here I will also tell you how to get the location of selected text (i.e. number of characters in the text view before the selected text) and also about accessing the length of selected text (i.e. number of characters selected) .

 CFRange selectedCFRange;
AXValueRef selectedRangeValue = NULL;

// Access selected range attribute from focused element
AXError selectedRangeError = AXUIElementCopyAttributeValue(focussedElement, kAXSelectedTextRangeAttribute, (CFTypeRef *)&selectedRangeValue);
if (selectedRangeError == kAXErrorSuccess)

{

NSLog(@”\nSelected Range: %@”,selectedRangeValue);

//Selected Range is retrieved successfully, then get the range into CFRange type object

AXValueGetValue(selectedRangeValue, kAXValueCFRangeType, &selectedCFRange);
}
else
{
NSLog(@”Error while retrieving selected range”);
}

// The length and location of the selected text will be selectedCFRange.length and selectedCFRange.location

NSLog(@”\nLength: %ld, Location: %ld”,selectedCFRange.length, selectedCFRange.location);

CGRect selectedRect;

AXValueRef selectedBounds = NULL;
// Get the selected bounds value from the selected range

AXError selectedBoundsError = AXUIElementCopyParameterizedAttributeValue(focussedElement, kAXBoundsForRangeParameterizedAttribute, selectedRangeValue, (CFTypeRef *)&selectedBounds);
CFRelease(selectedRangeValue);

if (selectedBoundsError == kAXErrorSuccess)

{

AXValueGetValue(selectedBounds, kAXValueCGRectType, &selectedRect);
NSLog(@”Selection bounds: %@”, NSStringFromRect(NSRectFromCGRect(selectedRect)));   // Selection Rect retrieved
}

else

{
NSLog(@”Error while retrieving selected range bounds”);
}

5. Remember to release the objects if not using ARC.

if (selectedBounds != NULL)

{

CFRelease(selectedBounds);

}

if (focussedElement != NULL)

{
CFRelease(focussedElement);

}
CFRelease(systemWideElement);

In this way you can use Accessibility API to perform various tasks and modify the above code to access different values from any currently active system wide applications

Written By: Neha Gupta, Software Engineer, Mindfire Solutions

Advertisements

About Neha Gupta

Software Engineer at Noida, India, Work on MAC OSX Application development with Cocoa, iOS development and Qt (Cross Platform Application Framework)

Posted on February 5, 2014, in Cocoa Application, Objective-C and tagged , , , , , , , , . Bookmark the permalink. 2 Comments.

  1. Reblogged this on Developers Area.

  2. salariegagnant

    This does not work with several applications like mail, chrome or Firefox. How could we fix that?

    Thanks and regards,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: