Open Radar

 
Summary:
The AXTextMarkerForPosition parameterized Accessibility attribute is broken in Safari 5.1, and almost always returns the text marker corresponding to the upper left corner of the current visible area rather than the position given.

Steps to Reproduce:
Compile and run this program:

// gcc -framework Cocoa -framework ApplicationServices axtest.m


#import <Cocoa/Cocoa.h>


#define ERR_CHECK( expr ) do { \
        AXError my_innerErr = expr; \
        if( my_innerErr ) \
        { \
            if(my_innerErr != kAXErrorAttributeUnsupported && \
			   my_innerErr != kAXErrorNotImplemented && \
               my_innerErr != kAXErrorNoValue) \
                NSLog( @"%s:%d: %s (%ld)", __func__, __LINE__, GetMacOSStatusErrorString( my_innerErr ), (long)my_innerErr ); \
            return nil; \
        } \
    } while( 0 )

static AXValueRef ValueForParameterizedAttribute(CFStringRef attr, CFTypeRef parameter, AXUIElementRef elt)
{
	CFTypeRef result;
	ERR_CHECK( AXUIElementCopyParameterizedAttributeValue( elt, attr, parameter, &result ) );
	return CFMakeCollectable( result );
}

static id ValueForParameter(CFStringRef attr, AXUIElementRef elt)
{
    CFTypeRef result;
    ERR_CHECK(AXUIElementCopyAttributeValue(elt, attr, &result));
    return [(id)result autorelease];
}

static id Test(void)
{
    AXUIElementRef sysElement = AXUIElementCreateSystemWide();
        
    while(1)
    {
        NSPoint p = [NSEvent mouseLocation];
        NSScreen *screen = [[NSScreen screens] objectAtIndex: 0];
        p.y = [screen frame].size.height - p.y - 1;
        
		AXValueRef ptObj = CFMakeCollectable( AXValueCreate( kAXValueCGPointType, &p ) );
		
        AXUIElementRef elt;
        ERR_CHECK( AXUIElementCopyElementAtPosition( sysElement, p.x, p.y, &elt ) );
        
        AXValueRef marker = ValueForParameterizedAttribute(CFSTR("AXTextMarkerForPosition"), ptObj, elt);
        if( !marker )
            goto webkitFail;
        
        AXValueRef endMarker = ValueForParameterizedAttribute(CFSTR("AXNextSentenceEndTextMarkerForTextMarker"), marker, elt);
        if( !endMarker )
            goto webkitFail;
        
        NSArray *markersArray = [NSArray arrayWithObjects: (id)marker, endMarker, nil];
        AXValueRef range = ValueForParameterizedAttribute(CFSTR("AXTextMarkerRangeForUnorderedTextMarkers"), (CFTypeRef)markersArray, elt);
        if( !range )
            goto webkitFail;
        
        NSString *endString = (id)ValueForParameterizedAttribute(CFSTR("AXStringForTextMarkerRange"), range, elt);
        if( !endString || ![endString isKindOfClass: [NSString class]] )
            goto webkitFail;
        
        AXValueRef startMarker = ValueForParameterizedAttribute(CFSTR("AXPreviousSentenceStartTextMarkerForTextMarker"), marker, elt);
        if( !startMarker )
            goto webkitFail;
        
        markersArray = [NSArray arrayWithObjects: (id)startMarker, endMarker, nil];
        range = ValueForParameterizedAttribute(CFSTR("AXTextMarkerRangeForUnorderedTextMarkers"), (CFTypeRef)markersArray, elt);
        if( !range )
            goto webkitFail;
        
        NSString *fullString = (id)ValueForParameterizedAttribute(CFSTR("AXStringForTextMarkerRange"), range, elt);
        if( !fullString || ![fullString isKindOfClass: [NSString class]] )
            goto webkitFail;
        
        NSLog(@"%@", fullString);
        
    webkitFail:
        ;
        
        sleep(1);
    }
}

int main(int argc, char **argv)
{
    [NSAutoreleasePool new];
    
    NSApplicationLoad();
    Test();
}

It will attempt to use AXTextMarkerForPosition to print the text under the mouse cursor. First, test it with an app that uses the original WebKit, such as Adium, Colloquy, NetNewsWire, Safari 5 or below, etc. Next, test it with Safari 5.1.

Expected Results:
The test app should print the text under the mouse cursor at all times.

Actual Results:
In Safari 5.1, it usually prints the text located at the top left corner of the current view. For example, I visited http://www.apple.com/mac/ in Safari 5.1 and ran this test program. As I scrolled down the page, the program continued to print whatever text was at that corner, ignoring my mouse position.

Regression:
This is new in Safari 5.1. It fails on both Lion and Snow Leopard.

Notes:
While AXTextMarkerForPosition is private API, it's the only way that I know of to extract the text at a specific point in Safari. Additionally, on Snow Leopard, the built-in dictionary lookup shortcut (cmd-control-d) uses this attribute, and consequently it no longer works in Safari 5.1.

If there is a better way to get the text at a specific point in Safari, I would love to hear it!