Open Radar

 
the provided readline compatible interface of libedit does not work properly for prompts containing ANSI color codes. Even if the non-printing characters are properly enclosed within RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE they are counted in the length calculation resulting in wrong assumptions on the beginning and end of editable line. This can be seen when jumping at the begin or end of entered line with ctrl-a / ctrl-e: the curser is placed to far right. Another way to see this is when browsing through the history: the first characters of the previous commands are not re-drawn.

Steps to Reproduce:
compile and start the attached sample using something like:
gcc -lreadline prompt.c -o prompt && ./prompt

enter 20-30 characters of text. Jump around with ctrl-a / ctrl-e: you see the cursor not positioned at the begin / end of the actual line. Press enter and enter several inputs of varying lengths. On an empty line use up/down keys. You see the history expanded starting at a position about 20 characters to far right. End the test program by entering "quit" or pressing ctrl-c

Expected Results:
install the original readline library:
brew install readline
compile the test program using that:
gcc -L/usr/local/opt/readline/lib -lreadline prompt.c -o prompt && ./prompt
perform the same tests as before. Start / End of the editable line is correctly detected

Actual Results:
Start / End of the editable line is not correctly detected if the prompt contains escaped non-printable codes. This mis-behaviour can be seen with several 3rd party tools compiled on macos, like the command line interface to PostgreSQL psql.

Version:
macOS Sierra 10.12.4 (16E195)
Xcode 8.3.2
$ otool -L ./prompt
./prompt:
	/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.50.2)


Attachment:
-----8<-- prompt.c -----8<--------
#include <readline/readline.h>
#include <readline/history.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define CYELLOW "\001\e[0;31m\002"
#define RESET   "\001\e[0m\002"
 
int main()
{
    char *buf;

    while((buf = readline(CYELLOW "prompt> " RESET))!=NULL)
    {
        if (strcmp(buf,"quit")==0)
            break;
 
        printf("[%s]\n",buf);
 
        if (buf[0]!=0)
            add_history(buf);
    }
 
    free(buf);
 
    return 0;
}
----->8-------------->8--------