cleaning genes database polluted by non-numeric characters except plus and minus signs

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I have this genes database which completely messed up by extra non-alphanumeric values. This happened as a sort of encryption to the data which was made incorrectly, and I don't know how to clean it up. I tried sed and awk, but failed. This is sample of the data which is very large amount of documents



chr2#@!!~//=^%$74711&&*&127472363@Pos1%%0^^+
chr3#@!!~//=^%$74723&&*&127473530@Pos2%%0^^+
chr1#@!!~//=^%$73530&&*&127474697@Pos3%%0^^+
chr2#@!!~//=^%$17469&&*&127475864@Pos4%%0^^+
chr3#@!!~//=^%$12747&&*&127477031@Neg1%%0^^-
chr5#@!!~//=^%$17477&&*&127478198@Neg2%%0^^-
chr7#@!!~//=^%$74781&&*&127479365@Neg3%%0^^-
chr7#@!!~//=^%$74795&&*&127480532@Pos5%%0^^+
chr1#@!!~//=^%$12748&&*&127481699@Neg4%%0^^-


The cleaned data must be like this 



chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -


please help very big problem here!






text-processing awk sed bioinformatics







share|improve this question









New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.



















  • try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
    – mosvy
    46 mins ago














up vote
1
down vote

favorite












I have this genes database which completely messed up by extra non-alphanumeric values. This happened as a sort of encryption to the data which was made incorrectly, and I don't know how to clean it up. I tried sed and awk, but failed. This is sample of the data which is very large amount of documents



chr2#@!!~//=^%$74711&&*&127472363@Pos1%%0^^+
chr3#@!!~//=^%$74723&&*&127473530@Pos2%%0^^+
chr1#@!!~//=^%$73530&&*&127474697@Pos3%%0^^+
chr2#@!!~//=^%$17469&&*&127475864@Pos4%%0^^+
chr3#@!!~//=^%$12747&&*&127477031@Neg1%%0^^-
chr5#@!!~//=^%$17477&&*&127478198@Neg2%%0^^-
chr7#@!!~//=^%$74781&&*&127479365@Neg3%%0^^-
chr7#@!!~//=^%$74795&&*&127480532@Pos5%%0^^+
chr1#@!!~//=^%$12748&&*&127481699@Neg4%%0^^-


The cleaned data must be like this 



chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -


please help very big problem here!






text-processing awk sed bioinformatics







share|improve this question









New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.



















  • try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
    – mosvy
    46 mins ago












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have this genes database which completely messed up by extra non-alphanumeric values. This happened as a sort of encryption to the data which was made incorrectly, and I don't know how to clean it up. I tried sed and awk, but failed. This is sample of the data which is very large amount of documents



chr2#@!!~//=^%$74711&&*&127472363@Pos1%%0^^+
chr3#@!!~//=^%$74723&&*&127473530@Pos2%%0^^+
chr1#@!!~//=^%$73530&&*&127474697@Pos3%%0^^+
chr2#@!!~//=^%$17469&&*&127475864@Pos4%%0^^+
chr3#@!!~//=^%$12747&&*&127477031@Neg1%%0^^-
chr5#@!!~//=^%$17477&&*&127478198@Neg2%%0^^-
chr7#@!!~//=^%$74781&&*&127479365@Neg3%%0^^-
chr7#@!!~//=^%$74795&&*&127480532@Pos5%%0^^+
chr1#@!!~//=^%$12748&&*&127481699@Neg4%%0^^-


The cleaned data must be like this 



chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -


please help very big problem here!






text-processing awk sed bioinformatics







share|improve this question









New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I have this genes database which completely messed up by extra non-alphanumeric values. This happened as a sort of encryption to the data which was made incorrectly, and I don't know how to clean it up. I tried sed and awk, but failed. This is sample of the data which is very large amount of documents



chr2#@!!~//=^%$74711&&*&127472363@Pos1%%0^^+
chr3#@!!~//=^%$74723&&*&127473530@Pos2%%0^^+
chr1#@!!~//=^%$73530&&*&127474697@Pos3%%0^^+
chr2#@!!~//=^%$17469&&*&127475864@Pos4%%0^^+
chr3#@!!~//=^%$12747&&*&127477031@Neg1%%0^^-
chr5#@!!~//=^%$17477&&*&127478198@Neg2%%0^^-
chr7#@!!~//=^%$74781&&*&127479365@Neg3%%0^^-
chr7#@!!~//=^%$74795&&*&127480532@Pos5%%0^^+
chr1#@!!~//=^%$12748&&*&127481699@Neg4%%0^^-


The cleaned data must be like this 



chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -


please help very big problem here!






text-processing awk sed bioinformatics




text-processing awk sed bioinformatics






share|improve this question









New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 14 mins ago









Jeff Schaller

33.6k851113




33.6k851113






New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 51 mins ago









marco

1024




1024




New contributor




marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






marco is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











  • try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
    – mosvy
    46 mins ago
















  • try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
    – mosvy
    46 mins ago















try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
– mosvy
46 mins ago




try sed -E 's/[^a-zA-Z0-9+-]+/ /g' file
– mosvy
46 mins ago










2 Answers
2






active

oldest

votes

















up vote
3
down vote



accepted










You can do it with sed. Something as follows:



sed 's/[^a-zA-Z0-9+-]/ /g' file | column -tc2

chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -





share|improve this answer
















  • 1




    THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
    – marco
    35 mins ago

















up vote
2
down vote













With tr, transliterating characters from the complement of the wanted set to spaces, and squeezing repeats:



$ tr -sc '[:alnum:][:space:]+-' ' ' < data
chr2 74711 127472363 Pos1 0 +
chr3 74723 127473530 Pos2 0 +
chr1 73530 127474697 Pos3 0 +
chr2 17469 127475864 Pos4 0 +
chr3 12747 127477031 Neg1 0 -
chr5 17477 127478198 Neg2 0 -
chr7 74781 127479365 Neg3 0 -
chr7 74795 127480532 Pos5 0 +
chr1 12748 127481699 Neg4 0 -





share|improve this answer






















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );






    marco is a new contributor. Be nice, and check out our Code of Conduct.









     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f473709%2fcleaning-genes-database-polluted-by-non-numeric-characters-except-plus-and-minus%23new-answer', 'question_page');

    );

    Post as a guest

















    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    3
    down vote



    accepted










    You can do it with sed. Something as follows:



    sed 's/[^a-zA-Z0-9+-]/ /g' file | column -tc2

    chr2 74711 127472363 Pos1 0 +
    chr3 74723 127473530 Pos2 0 +
    chr1 73530 127474697 Pos3 0 +
    chr2 17469 127475864 Pos4 0 +
    chr3 12747 127477031 Neg1 0 -
    chr5 17477 127478198 Neg2 0 -
    chr7 74781 127479365 Neg3 0 -
    chr7 74795 127480532 Pos5 0 +
    chr1 12748 127481699 Neg4 0 -





    share|improve this answer
















    • 1




      THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
      – marco
      35 mins ago














    up vote
    3
    down vote



    accepted










    You can do it with sed. Something as follows:



    sed 's/[^a-zA-Z0-9+-]/ /g' file | column -tc2

    chr2 74711 127472363 Pos1 0 +
    chr3 74723 127473530 Pos2 0 +
    chr1 73530 127474697 Pos3 0 +
    chr2 17469 127475864 Pos4 0 +
    chr3 12747 127477031 Neg1 0 -
    chr5 17477 127478198 Neg2 0 -
    chr7 74781 127479365 Neg3 0 -
    chr7 74795 127480532 Pos5 0 +
    chr1 12748 127481699 Neg4 0 -





    share|improve this answer
















    • 1




      THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
      – marco
      35 mins ago












    up vote
    3
    down vote



    accepted







    up vote
    3
    down vote



    accepted






    You can do it with sed. Something as follows:



    sed 's/[^a-zA-Z0-9+-]/ /g' file | column -tc2

    chr2 74711 127472363 Pos1 0 +
    chr3 74723 127473530 Pos2 0 +
    chr1 73530 127474697 Pos3 0 +
    chr2 17469 127475864 Pos4 0 +
    chr3 12747 127477031 Neg1 0 -
    chr5 17477 127478198 Neg2 0 -
    chr7 74781 127479365 Neg3 0 -
    chr7 74795 127480532 Pos5 0 +
    chr1 12748 127481699 Neg4 0 -





    share|improve this answer












    You can do it with sed. Something as follows:



    sed 's/[^a-zA-Z0-9+-]/ /g' file | column -tc2

    chr2 74711 127472363 Pos1 0 +
    chr3 74723 127473530 Pos2 0 +
    chr1 73530 127474697 Pos3 0 +
    chr2 17469 127475864 Pos4 0 +
    chr3 12747 127477031 Neg1 0 -
    chr5 17477 127478198 Neg2 0 -
    chr7 74781 127479365 Neg3 0 -
    chr7 74795 127480532 Pos5 0 +
    chr1 12748 127481699 Neg4 0 -






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered 47 mins ago









    Goro

    6,94352965




    6,94352965







    • 1




      THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
      – marco
      35 mins ago












    • 1




      THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
      – marco
      35 mins ago







    1




    1




    THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
    – marco
    35 mins ago




    THIS IS AMAZING!!!!!!!!!!!!! thank you @goro i didn't know how to keep - and +
    – marco
    35 mins ago












    up vote
    2
    down vote













    With tr, transliterating characters from the complement of the wanted set to spaces, and squeezing repeats:



    $ tr -sc '[:alnum:][:space:]+-' ' ' < data
    chr2 74711 127472363 Pos1 0 +
    chr3 74723 127473530 Pos2 0 +
    chr1 73530 127474697 Pos3 0 +
    chr2 17469 127475864 Pos4 0 +
    chr3 12747 127477031 Neg1 0 -
    chr5 17477 127478198 Neg2 0 -
    chr7 74781 127479365 Neg3 0 -
    chr7 74795 127480532 Pos5 0 +
    chr1 12748 127481699 Neg4 0 -





    share|improve this answer


























      up vote
      2
      down vote













      With tr, transliterating characters from the complement of the wanted set to spaces, and squeezing repeats:



      $ tr -sc '[:alnum:][:space:]+-' ' ' < data
      chr2 74711 127472363 Pos1 0 +
      chr3 74723 127473530 Pos2 0 +
      chr1 73530 127474697 Pos3 0 +
      chr2 17469 127475864 Pos4 0 +
      chr3 12747 127477031 Neg1 0 -
      chr5 17477 127478198 Neg2 0 -
      chr7 74781 127479365 Neg3 0 -
      chr7 74795 127480532 Pos5 0 +
      chr1 12748 127481699 Neg4 0 -





      share|improve this answer
























        up vote
        2
        down vote










        up vote
        2
        down vote









        With tr, transliterating characters from the complement of the wanted set to spaces, and squeezing repeats:



        $ tr -sc '[:alnum:][:space:]+-' ' ' < data
        chr2 74711 127472363 Pos1 0 +
        chr3 74723 127473530 Pos2 0 +
        chr1 73530 127474697 Pos3 0 +
        chr2 17469 127475864 Pos4 0 +
        chr3 12747 127477031 Neg1 0 -
        chr5 17477 127478198 Neg2 0 -
        chr7 74781 127479365 Neg3 0 -
        chr7 74795 127480532 Pos5 0 +
        chr1 12748 127481699 Neg4 0 -





        share|improve this answer














        With tr, transliterating characters from the complement of the wanted set to spaces, and squeezing repeats:



        $ tr -sc '[:alnum:][:space:]+-' ' ' < data
        chr2 74711 127472363 Pos1 0 +
        chr3 74723 127473530 Pos2 0 +
        chr1 73530 127474697 Pos3 0 +
        chr2 17469 127475864 Pos4 0 +
        chr3 12747 127477031 Neg1 0 -
        chr5 17477 127478198 Neg2 0 -
        chr7 74781 127479365 Neg3 0 -
        chr7 74795 127480532 Pos5 0 +
        chr1 12748 127481699 Neg4 0 -






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited 16 mins ago

























        answered 26 mins ago









        steeldriver

        32.5k34980




        32.5k34980




















            marco is a new contributor. Be nice, and check out our Code of Conduct.









             

            draft saved


            draft discarded


















            marco is a new contributor. Be nice, and check out our Code of Conduct.












            marco is a new contributor. Be nice, and check out our Code of Conduct.











            marco is a new contributor. Be nice, and check out our Code of Conduct.













             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f473709%2fcleaning-genes-database-polluted-by-non-numeric-characters-except-plus-and-minus%23new-answer', 'question_page');

            );

            Post as a guest
































































            Comments

            Post a Comment

            Popular posts from this blog

            What does second last employer means? [closed]

            Image
            Clash Royale CLAN TAG #URR8PPP .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0; up vote -3 down vote favorite Say, for example I have worked in the following companies, Company A => 2010 - 2011 Company B => 2011 - 2012 Company C => 2012 - Till Date Company D => Have offer Now assume Company D is asking to submit documents from my second last employer. So from the list of companies given, which company would be qualified as second last employer? software-industry employer share | improve this question asked Aug 10 '14 at 5:16 Rajaprabhu Aravindasamy 559 1 6 13 closed as off-topic by Jim G., jmort253 ♦ Aug 11 '14 at 1:58 This question appears to be off-topic. The users who voted to close gave this specific reason: "Questions seeking advice on company-specific regulations, agreements, or policies should be directed to your manager or HR department. Questions that address...
            Read more

            Installing NextGIS Connect into QGIS 3?

            Image
            Clash Royale CLAN TAG #URR8PPP .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0; up vote 1 down vote favorite Just acquired free version of NEXTGIS and trying to install the plugin for QGIS. It tells me it was not designed for version 3 of QGIS. How do I set up the QGIS-NEXTGIS interface then? qgis qgis-plugins nextgis share | improve this question edited 3 hours ago PolyGeo ♦ 51.9k 17 77 233 asked 4 hours ago IanGS 16 4 New contributor IanGS is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct. add a comment  |  up vote 1 down vote favorite Just acquired free version of NEXTGIS and trying to install the plugin for QGIS. It tells me it was not designed for version 3 of QGIS. How do I set up the QGIS-NEXTGIS interface then? qgis qgis-plugin...
            Read more

            One-line joke

            A one-liner is a joke that is delivered in a single line. A good one-liner is said to be pithy - concise and meaningful. [1] Comedians and actors use this comedic method as part of their act, e.g. Jimmy Carr, Tommy Cooper, Rodney Dangerfield, Norm Macdonald, Ken Dodd, Stewart Francis, Zach Galifianakis, Mitch Hedberg, Anthony Jeselnik, Milton Jones, Shappi Khorsandi, Jay London, Mark Linn-Baker, Demetri Martin, Groucho Marx, Dan Mintz, Emo Philips, Tim Vine, Steven Wright and Henny Youngman and Arnold Schwarzenegger. [ citation needed ] Many fictional characters are also known to deliver one-liners, including James Bond, who usually includes pithy and laconic quips after disposing of a villain. [ citation needed ] Examples "A baby seal walks into a club." "A dyslexic man walks into a bra." (Anonymous) "There are three types of people, those who can count and those who can't." "Venison's dear, isn't it?" (Jimmy Carr) "An escala...
            Read more