Open Radar

 
Summary:

NSEvent's -mouseLocation method returns invalid coordinates for Retina backing store


Steps to Reproduce:

1) Obtain the current mouse location using [NSEvent mouseLocation]
2) Align the mouse location to a screen's backing store using NSScreen's -backingAlignedRect:options:
3) Convert the aligned mouse location to backing store coordinates with NSScreen's -convertRectToBacking:

See the attached CoordinateTester app as a sample.


Expected Results:

The backing store coordinates should be valid.


Actual Results:

The backing store coordinates are invalid in the Y axis. On a 15" Retina the range of values should be 0 to 1799 (1800 physical pixels), instead a range of 0 to 1800 pixels is reported (1801 pixels). Similarly, on non-Retina displays such as the 13" MacBook Air, a range of values 0 to 900 is reported (901 pixels).


Regression:

In prior releases of Mac OS X, the mouse location's Y axis was based at one, so a correct coordinate could be obtained by applying an offset. Values less than one are now returned (see Radar # 11905218)


Notes:

Running the attached CoordinateTester app on several different systems shows these incorrect results are consistent:

									Raw				Raw						Aligned		Aligned			Backing
							OS		Lower-left		Upper-right				Lower-left	Upper-right		Upper-right
							----	----------		-----------				----------	-----------		-----------
MacBook Pro (15" Retina)	10.8	0, 0.00390625	1439.99609375, 900		0, 0		1439.5, 900		2879, 1800
MacBook Air (13")			10.8	0, 0.00390625	1439.99609375, 900		0, 0		1439, 900		1439, 900
MacBook Air (13")			10.7	0, 0.00390625	1439.99609375, 900		0, 0		1439, 900		1439, 900
MacPro (2 x 1600 x 2560)	10.7	0, 0.00390625	1439.99609375, 900		0, 0		3199, 2560		3199, 2560

Note: Raw lower-left Y coordinate is off by (1 / 256). The raw upper-right X coordinate is off by ((height - (1 / 256)) - 1).


Additionally, one would expect mouse coordinate deltas to correspond with the current backing store scaling factor. For example:

								Screen								Backing
				Screen Size		scaling factor		Backing Size	scaling factor
				-----------		--------------		------------	--------------
Larger Text:	1024 x 640		0.3556				2048 x 1280		0.7111
				1280 x 800		0.4444				2560 x 1600		0.8889
Best:			1440 x 900		0.5					2880 x 1800		1.0
				1680 x 1050		0.5833				3360 x 2100		1.1667
More Space:		1920 x 1200		0.6667				3840 x 2400		1.3333

Native:			2880 x 1800							2880 x 1800		

That is, as the user moves the mouse one pixel on the physical screen of 2880 x 1800, mouse location values should be reported in 0.5 increments in the scaled 1440 x 900 view coordinate system. Similarly, if a user has 1800 physical pixels scaled to 1200 view points, there should be 1800 unique positions reported that are 0.6667 units apart.

Not only is the mouse location distribution nonuniform, there's also quite a bit of variance in the values that are reported. For example, the following values were returned when the backing store Y axis was 1.0:

Raw mouse	Delta from origin
Y axis		at 0.00390625
---------	-----------------
0.515625	0.51171875
0.6015625	0.59765625
0.6953125	0.69140625
0.9609375	0.95703125

Finally, it's *really* hard to move the mouse to point where the backing Y axis is 1.0. Even when there are more physical pixels than view points (such as the 1024 x 640 and 1280 x 800 settings.) Try and get the CoordinateTester app to trigger the NSLog() and you'll see what I mean.


So, am I just being pedantic here?

I don't think so: the output resolution of Retina display (and the underlying changes to Mac OS X) are an impressive achievement. Applications that wish to use that resolution to the fullest will also need an accurate input resolution. Mouse, trackpad and drawing tablet inputs should be able to address any pixel that's moved from the backing store onto the display.

Currently, that's not possible.

A good example is the Pixie developer tool that's used to sample screen colors. It's impossible to sample adjacent pixels, no matter what the setting is for display scaling. Sampling on-screen pixels is an essential part of many designers and developers workflows when building and testing applications.

I can also envision problems with applications that want to do precision drawing into the backing store: it can't be done without a precise point to position the drawing.