Accessing text value from any System wide Application via Accessibility API

Suppose, I want to create a application which will monitor the typing and say if “macdevelopers” is typed anywhere be it in TextEdit or Mail etc. the application will perform a operation for example automatically opening the website in a browser.

The text value from any System wide Application can be accessed using the Accessibility API. In this article I will discuss about accessing the text value from any application like Text Edit or other applications if its current focused element is some text field or text view in it. Thus using this you can implement the functionality to access the active application’s text field value.

First of all, for using Accessibility API you must have enabled access for assistive devices in Universal Access preference pane in System Preferences.

Mark the check box “Enable Access for assistive devices” in Universal Access Preferences pane. (Note that in OS X 10.8 Universal Access pane has been renamed to Accessibility )

To programmatically enable access for assistive devices you may create a Apple Script file with the script below and execute this script from your application.

do shell script “touch /private/var/db/.AccessibilityAPIEnabled” with administrator privileges

Now, to access the text value from the current focused element of any application, follow the steps below:

1. Create a system wide element to retrieve the focussed element of any application

AXUIElementRef systemWideElement = AXUIElementCreateSystemWide();    //creating system wide element

2. Retrieve the focussed element using kAXFocusedUIElementAttribute. Refer Apple’s documentation for other attributes.

AXUIElementRef focussedElement = NULL;

AXError error = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute, (CFTypeRef *)&focussedElement);                //Copy the focused element from the systemWideElement to focussedElement

if (error != kAXErrorSuccess)           // if focused element could not be retrieved

{

NSLog(@”Could not get focused element”);

}

3. Now, if the focused element is retrieved, we need to access its string value.

else

{

AXValueRef textValue = NULL;

AXError gettextvalueError = AXUIElementCopyAttributeValue(focussedElement, kAXValueAttribute , (CFTypeRef *)&textValue);   //get the text value of focussedElement in object textValue

NSString* stringVal = (NSString*)textValue;      // Convert textValue to NSString

NSLog(@”%@”, stringVal);

}

In this way you can use Accessibility API to accomplish various system wide tasks.

Written By: Neha Gupta, Software Engineer, Mindfire Solutions

Advertisements

About Neha Gupta

Software Engineer at Noida, India, Work on MAC OSX Application development with Cocoa, iOS development and Qt (Cross Platform Application Framework)

Posted on January 31, 2014, in Cocoa Application, Objective-C and tagged , , , , , , , , , , . Bookmark the permalink. 4 Comments.

  1. Reblogged this on Developers Area.

  2. Hello, thanks for the article. But the code above executed once and get the selected text at startup, how can to detect selected text if user open new application and selecting text in it?

  3. Hello Zillion,
    You may execute the above code on some event (like some global key down event or the application open event) to get the text value from focused text element that particular time.

  4. Hi Neha,
    Can you please tell me a way to figure out if an element is focused while Voice Over is ON in an OS X application. We have a function in iOS “accessibilityElementDidBecomeFocused” which tells us that but I am not able to find out a similar thing is OS X.

    Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: