How can I categorize files in a directory based on their content?Merge CSV files with field delimiters also occuring inside quotesSwitch Columns in .csv files so that they are all the samePick columns from a variable length csv fileHow to extract column name (header) from a CSV file which contains the max value in a row?How can I combine different csv files from differenet folders with the name of each file as a column name in the combined file?Finding average excluding the first rowHow to use Unix Shell to show only the first n columns and last n columns?Empty multiple .csv log files but retain the headerSorting the CSV based on Unix timestampJoining / merging CSV files that do not share all their headers / column

How to remove empty lines from begin and end of file?

Where is the node created timestamp stored in the database?

Is there any math conjecture that would cause a lot of damage if disproven?

Outlining the climax made me lose interest in writing the actual story

Why does Rome municipality seem to have a hard time maintaining the city?

In an interview, is it self-defeating to say you use StackOverflow to find errors in code?

Does a resurrected wizard remember his prepared spells?

Sort and Table a Sentence by Word Lengths

Were any DOS games (or software) known to use VBE/AF?

If a stored procedure is altered while it's currently being used in a cursor, does the cursor continue using the old query before it was altered?

Why does Greedo say "Maclunkey" in the Mos Eisley Cantina?

Co-curricular lessons between geometry and chemistry?

How to subtract two different values from two different columns and print if less than a value?

Can increasing the amount of training data make overfitting worse?

When to use Sitecore.Context.Items and why?

Why is there so much dispute about how ornaments are supposed to be played in classical music?

What is the maximum distance you can cause damage from?

Can a microwave oven cook chicken?

Unique magic triplets

How do you create an APFS volume inside an ordinary file?

Why don't we shield existing CPUs from radiation instead of designing new ones?

A robot surviving on top of a 3x3 platform

What should I tell a customer when my co-worker fails to show up to a meeting?

Grid Puzzle - Paint



How can I categorize files in a directory based on their content?


Merge CSV files with field delimiters also occuring inside quotesSwitch Columns in .csv files so that they are all the samePick columns from a variable length csv fileHow to extract column name (header) from a CSV file which contains the max value in a row?How can I combine different csv files from differenet folders with the name of each file as a column name in the combined file?Finding average excluding the first rowHow to use Unix Shell to show only the first n columns and last n columns?Empty multiple .csv log files but retain the headerSorting the CSV based on Unix timestampJoining / merging CSV files that do not share all their headers / column






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;









2


















I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).



I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:



YYYY/
└── MM/
└── DD/


Is it possible? How can I do that?



Content of CSV file is like below:



timestamp,A,B,C,D,E,F,G,H,I


for example:



1565592149,A,B,C,D,E,F,G,H,I









share|improve this question

































    2


















    I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).



    I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:



    YYYY/
    └── MM/
    └── DD/


    Is it possible? How can I do that?



    Content of CSV file is like below:



    timestamp,A,B,C,D,E,F,G,H,I


    for example:



    1565592149,A,B,C,D,E,F,G,H,I









    share|improve this question





























      2













      2









      2


      1






      I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).



      I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:



      YYYY/
      └── MM/
      └── DD/


      Is it possible? How can I do that?



      Content of CSV file is like below:



      timestamp,A,B,C,D,E,F,G,H,I


      for example:



      1565592149,A,B,C,D,E,F,G,H,I









      share|improve this question
















      I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).



      I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:



      YYYY/
      └── MM/
      └── DD/


      Is it possible? How can I do that?



      Content of CSV file is like below:



      timestamp,A,B,C,D,E,F,G,H,I


      for example:



      1565592149,A,B,C,D,E,F,G,H,I






      bash csv time






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Aug 12 at 7:37









      Stéphane Chazelas

      344k59 gold badges673 silver badges1051 bronze badges




      344k59 gold badges673 silver badges1051 bronze badges










      asked Aug 12 at 6:10









      SRFSRF

      1314 bronze badges




      1314 bronze badges























          2 Answers
          2






          active

          oldest

          votes


















          10



















          Maybe something like:



          #! /bin/bash -
          for f in *.csv; do
          IFS=, read -r timestamp rest < "$f" &&
          printf -v dir '%(%Y/%m/%d)T' "$timestamp" &&
          mkdir -p -- "$dir" &&
          mv -- "$f" "$dir/"
          done


          Example:



          $ head -- *.csv
          ==> test2.csv <==
          1328012580,A,B,C,D,E,F,G,H,I

          ==> test.csv <==
          1565592149,A,B,C,D,E,F,G,H,I
          $ that-script
          $ tree
          .
          ├── 2012
          │   └── 01
          │   └── 31
          │   └── test2.csv
          └── 2019
          └── 08
          └── 12
          └── test.csv

          6 directories, 2 files





          share|improve this answer




























          • Nice! (Requires Bash 4.2 or later.)

            – Dennis Williamson
            Aug 12 at 16:21


















          2



















          To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.



          For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh



          • you could then run crontab -e to edit a the users crontab

          • At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh

          This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.






          share|improve this answer



























            Your Answer








            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "106"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );














            draft saved

            draft discarded
















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f535111%2fhow-can-i-categorize-files-in-a-directory-based-on-their-content%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown


























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            10



















            Maybe something like:



            #! /bin/bash -
            for f in *.csv; do
            IFS=, read -r timestamp rest < "$f" &&
            printf -v dir '%(%Y/%m/%d)T' "$timestamp" &&
            mkdir -p -- "$dir" &&
            mv -- "$f" "$dir/"
            done


            Example:



            $ head -- *.csv
            ==> test2.csv <==
            1328012580,A,B,C,D,E,F,G,H,I

            ==> test.csv <==
            1565592149,A,B,C,D,E,F,G,H,I
            $ that-script
            $ tree
            .
            ├── 2012
            │   └── 01
            │   └── 31
            │   └── test2.csv
            └── 2019
            └── 08
            └── 12
            └── test.csv

            6 directories, 2 files





            share|improve this answer




























            • Nice! (Requires Bash 4.2 or later.)

              – Dennis Williamson
              Aug 12 at 16:21















            10



















            Maybe something like:



            #! /bin/bash -
            for f in *.csv; do
            IFS=, read -r timestamp rest < "$f" &&
            printf -v dir '%(%Y/%m/%d)T' "$timestamp" &&
            mkdir -p -- "$dir" &&
            mv -- "$f" "$dir/"
            done


            Example:



            $ head -- *.csv
            ==> test2.csv <==
            1328012580,A,B,C,D,E,F,G,H,I

            ==> test.csv <==
            1565592149,A,B,C,D,E,F,G,H,I
            $ that-script
            $ tree
            .
            ├── 2012
            │   └── 01
            │   └── 31
            │   └── test2.csv
            └── 2019
            └── 08
            └── 12
            └── test.csv

            6 directories, 2 files





            share|improve this answer




























            • Nice! (Requires Bash 4.2 or later.)

              – Dennis Williamson
              Aug 12 at 16:21













            10















            10











            10









            Maybe something like:



            #! /bin/bash -
            for f in *.csv; do
            IFS=, read -r timestamp rest < "$f" &&
            printf -v dir '%(%Y/%m/%d)T' "$timestamp" &&
            mkdir -p -- "$dir" &&
            mv -- "$f" "$dir/"
            done


            Example:



            $ head -- *.csv
            ==> test2.csv <==
            1328012580,A,B,C,D,E,F,G,H,I

            ==> test.csv <==
            1565592149,A,B,C,D,E,F,G,H,I
            $ that-script
            $ tree
            .
            ├── 2012
            │   └── 01
            │   └── 31
            │   └── test2.csv
            └── 2019
            └── 08
            └── 12
            └── test.csv

            6 directories, 2 files





            share|improve this answer
















            Maybe something like:



            #! /bin/bash -
            for f in *.csv; do
            IFS=, read -r timestamp rest < "$f" &&
            printf -v dir '%(%Y/%m/%d)T' "$timestamp" &&
            mkdir -p -- "$dir" &&
            mv -- "$f" "$dir/"
            done


            Example:



            $ head -- *.csv
            ==> test2.csv <==
            1328012580,A,B,C,D,E,F,G,H,I

            ==> test.csv <==
            1565592149,A,B,C,D,E,F,G,H,I
            $ that-script
            $ tree
            .
            ├── 2012
            │   └── 01
            │   └── 31
            │   └── test2.csv
            └── 2019
            └── 08
            └── 12
            └── test.csv

            6 directories, 2 files






            share|improve this answer















            share|improve this answer




            share|improve this answer








            edited Aug 12 at 7:26

























            answered Aug 12 at 7:18









            Stéphane ChazelasStéphane Chazelas

            344k59 gold badges673 silver badges1051 bronze badges




            344k59 gold badges673 silver badges1051 bronze badges















            • Nice! (Requires Bash 4.2 or later.)

              – Dennis Williamson
              Aug 12 at 16:21

















            • Nice! (Requires Bash 4.2 or later.)

              – Dennis Williamson
              Aug 12 at 16:21
















            Nice! (Requires Bash 4.2 or later.)

            – Dennis Williamson
            Aug 12 at 16:21





            Nice! (Requires Bash 4.2 or later.)

            – Dennis Williamson
            Aug 12 at 16:21













            2



















            To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.



            For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh



            • you could then run crontab -e to edit a the users crontab

            • At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh

            This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.






            share|improve this answer






























              2



















              To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.



              For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh



              • you could then run crontab -e to edit a the users crontab

              • At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh

              This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.






              share|improve this answer




























                2















                2











                2









                To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.



                For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh



                • you could then run crontab -e to edit a the users crontab

                • At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh

                This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.






                share|improve this answer














                To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.



                For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh



                • you could then run crontab -e to edit a the users crontab

                • At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh

                This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.







                share|improve this answer













                share|improve this answer




                share|improve this answer










                answered Aug 12 at 17:55









                William YoungWilliam Young

                211 bronze badge




                211 bronze badge































                    draft saved

                    draft discarded















































                    Thanks for contributing an answer to Unix & Linux Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f535111%2fhow-can-i-categorize-files-in-a-directory-based-on-their-content%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown









                    Popular posts from this blog

                    Tamil (spriik) Luke uk diar | Nawigatjuun

                    Align equal signs while including text over equalitiesAMS align: left aligned text/math plus multicolumn alignmentMultiple alignmentsAligning equations in multiple placesNumbering and aligning an equation with multiple columnsHow to align one equation with another multline equationUsing \ in environments inside the begintabularxNumber equations and preserving alignment of equal signsHow can I align equations to the left and to the right?Double equation alignment problem within align enviromentAligned within align: Why are they right-aligned?

                    Training a classifier when some of the features are unknownWhy does Gradient Boosting regression predict negative values when there are no negative y-values in my training set?How to improve an existing (trained) classifier?What is effect when I set up some self defined predisctor variables?Why Matlab neural network classification returns decimal values on prediction dataset?Fitting and transforming text data in training, testing, and validation setsHow to quantify the performance of the classifier (multi-class SVM) using the test data?How do I control for some patients providing multiple samples in my training data?Training and Test setTraining a convolutional neural network for image denoising in MatlabShouldn't an autoencoder with #(neurons in hidden layer) = #(neurons in input layer) be “perfect”?