I just want to shed some light on the issue. Taken from 
get_floor_item_index_under_mouse_cursor in items.c:

// In the case that X was pressed, we don't use the item positions but rather
// we use the item slot rectangles from the item texts.

[Further down]

// If no X was pressed, we only use the floor position the mouse
// has pointed to and see if we can find an item that has geographically
// that very same (or a similar enough) position.

This would indicate that the behavior is intentional. However, when testing this 
bug while holding 'x' I've found that clicking on the item's label is not always 
sufficient. I did this testing on r5066.