Bsrgvty

How to remove empty lines from begin and end of file?

Where is the node created timestamp stored in the database?

Is there any math conjecture that would cause a lot of damage if disproven?

Outlining the climax made me lose interest in writing the actual story

Why does Rome municipality seem to have a hard time maintaining the city?

In an interview, is it self-defeating to say you use StackOverflow to find errors in code?

Does a resurrected wizard remember his prepared spells?

Sort and Table a Sentence by Word Lengths

Were any DOS games (or software) known to use VBE/AF?

If a stored procedure is altered while it's currently being used in a cursor, does the cursor continue using the old query before it was altered?

Why does Greedo say "Maclunkey" in the Mos Eisley Cantina?

Co-curricular lessons between geometry and chemistry?

How to subtract two different values from two different columns and print if less than a value?

Can increasing the amount of training data make overfitting worse?

When to use Sitecore.Context.Items and why?

Why is there so much dispute about how ornaments are supposed to be played in classical music?

What is the maximum distance you can cause damage from?

Can a microwave oven cook chicken?

Unique magic triplets

How do you create an APFS volume inside an ordinary file?

Why don't we shield existing CPUs from radiation instead of designing new ones?

A robot surviving on top of a 3x3 platform

What should I tell a customer when my co-worker fails to show up to a meeting?

Grid Puzzle - Paint

How can I categorize files in a directory based on their content?

Merge CSV files with field delimiters also occuring inside quotesSwitch Columns in .csv files so that they are all the samePick columns from a variable length csv fileHow to extract column name (header) from a CSV file which contains the max value in a row?How can I combine different csv files from differenet folders with the name of each file as a column name in the combined file?Finding average excluding the first rowHow to use Unix Shell to show only the first n columns and last n columns?Empty multiple .csv log files but retain the headerSorting the CSV based on Unix timestampJoining / merging CSV files that do not share all their headers / column

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;

2

1

I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).

I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:

YYYY/
 └── MM/
 └── DD/

Is it possible? How can I do that?

Content of CSV file is like below:

timestamp,A,B,C,D,E,F,G,H,I

for example:

1565592149,A,B,C,D,E,F,G,H,I

bash csv time

share|improve this question

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

add a comment
 | 

2

1

I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).

I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:

YYYY/
 └── MM/
 └── DD/

Is it possible? How can I do that?

Content of CSV file is like below:

timestamp,A,B,C,D,E,F,G,H,I

for example:

1565592149,A,B,C,D,E,F,G,H,I

bash csv time

share|improve this question

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

add a comment
 | 

I have a directory which contains a lot of CSV files. The CSV files have many columns the first of which is a timestamp (as number of seconds since the UNIX Epoch). I want to categorize files in the directory based on the value of that timestamp column in the first line of each file. (There is no header row in the files).

I want a bash script that run on the directory every two minutes and categorize files in sub-directories in the following layout:

YYYY/
 └── MM/
 └── DD/

Is it possible? How can I do that?

Content of CSV file is like below:

timestamp,A,B,C,D,E,F,G,H,I

for example:

1565592149,A,B,C,D,E,F,G,H,I

bash csv time

share|improve this question

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

YYYY/
 └── MM/
 └── DD/

timestamp,A,B,C,D,E,F,G,H,I

1565592149,A,B,C,D,E,F,G,H,I

share|improve this question

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

share|improve this question

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

edited Aug 12 at 7:37

Stéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

asked Aug 12 at 6:10

SRFSRF

13144 bronze badges

2 Answers
2

active

oldest

votes

10

Maybe something like:

#! /bin/bash -
for f in *.csv; do
 IFS=, read -r timestamp rest < "$f" &&
 printf -v dir '%(%Y/%m/%d)T' "$timestamp" && 
 mkdir -p -- "$dir" &&
 mv -- "$f" "$dir/"
done

Example:

$ head -- *.csv
==> test2.csv <==
1328012580,A,B,C,D,E,F,G,H,I

==> test.csv <==
1565592149,A,B,C,D,E,F,G,H,I
$ that-script
$ tree
.
├── 2012
│   └── 01
│   └── 31
│   └── test2.csv
└── 2019
 └── 08
 └── 12
 └── test.csv

6 directories, 2 files

share|improve this answer

edited Aug 12 at 7:26

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Nice! (Requires Bash 4.2 or later.)
  
  – Dennis Williamson
  Aug 12 at 16:21
  

  
  
  
  

add a comment
 | 

2

To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.

For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh

• you could then run crontab -e to edit a the users crontab


• At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh
  

This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.

share|improve this answer

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

add a comment
 | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f535111%2fhow-can-i-categorize-files-in-a-directory-based-on-their-content%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

10

Maybe something like:

#! /bin/bash -
for f in *.csv; do
 IFS=, read -r timestamp rest < "$f" &&
 printf -v dir '%(%Y/%m/%d)T' "$timestamp" && 
 mkdir -p -- "$dir" &&
 mv -- "$f" "$dir/"
done

Example:

$ head -- *.csv
==> test2.csv <==
1328012580,A,B,C,D,E,F,G,H,I

==> test.csv <==
1565592149,A,B,C,D,E,F,G,H,I
$ that-script
$ tree
.
├── 2012
│   └── 01
│   └── 31
│   └── test2.csv
└── 2019
 └── 08
 └── 12
 └── test.csv

6 directories, 2 files

share|improve this answer

edited Aug 12 at 7:26

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Nice! (Requires Bash 4.2 or later.)
  
  – Dennis Williamson
  Aug 12 at 16:21
  

  
  
  
  

add a comment
 | 

10

Maybe something like:

#! /bin/bash -
for f in *.csv; do
 IFS=, read -r timestamp rest < "$f" &&
 printf -v dir '%(%Y/%m/%d)T' "$timestamp" && 
 mkdir -p -- "$dir" &&
 mv -- "$f" "$dir/"
done

Example:

$ head -- *.csv
==> test2.csv <==
1328012580,A,B,C,D,E,F,G,H,I

==> test.csv <==
1565592149,A,B,C,D,E,F,G,H,I
$ that-script
$ tree
.
├── 2012
│   └── 01
│   └── 31
│   └── test2.csv
└── 2019
 └── 08
 └── 12
 └── test.csv

6 directories, 2 files

share|improve this answer

edited Aug 12 at 7:26

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Nice! (Requires Bash 4.2 or later.)
  
  – Dennis Williamson
  Aug 12 at 16:21
  

  
  
  
  

add a comment
 | 

Maybe something like:

#! /bin/bash -
for f in *.csv; do
 IFS=, read -r timestamp rest < "$f" &&
 printf -v dir '%(%Y/%m/%d)T' "$timestamp" && 
 mkdir -p -- "$dir" &&
 mv -- "$f" "$dir/"
done

Example:

$ head -- *.csv
==> test2.csv <==
1328012580,A,B,C,D,E,F,G,H,I

==> test.csv <==
1565592149,A,B,C,D,E,F,G,H,I
$ that-script
$ tree
.
├── 2012
│   └── 01
│   └── 31
│   └── test2.csv
└── 2019
 └── 08
 └── 12
 └── test.csv

6 directories, 2 files

share|improve this answer

edited Aug 12 at 7:26

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

#! /bin/bash -
for f in *.csv; do
 IFS=, read -r timestamp rest < "$f" &&
 printf -v dir '%(%Y/%m/%d)T' "$timestamp" && 
 mkdir -p -- "$dir" &&
 mv -- "$f" "$dir/"
done

$ head -- *.csv
==> test2.csv <==
1328012580,A,B,C,D,E,F,G,H,I

==> test.csv <==
1565592149,A,B,C,D,E,F,G,H,I
$ that-script
$ tree
.
├── 2012
│   └── 01
│   └── 31
│   └── test2.csv
└── 2019
 └── 08
 └── 12
 └── test.csv

6 directories, 2 files

share|improve this answer

edited Aug 12 at 7:26

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

answered Aug 12 at 7:18

Stéphane ChazelasStéphane Chazelas

344k5959 gold badges673673 silver badges10511051 bronze badges

2

To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.

For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh

• you could then run crontab -e to edit a the users crontab


• At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh
  

This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.

share|improve this answer

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

add a comment
 | 

2

To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.

For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh

• you could then run crontab -e to edit a the users crontab


• At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh
  

This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.

share|improve this answer

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

add a comment
 | 

To accomplish the 'every 2 minutes' part of you question, you can put a script like the one Stephane Chazelas made, and invoke it using a cron job.

For example, if your CSV files were at /home/user/data and in that folder you have the script in script.sh

• you could then run crontab -e to edit a the users crontab


• At the end of the file you would add */2 * * * * cd /home/user/data && /home/user/data/script.sh
  

This would cause the script to be run every 2 minutes, if you wanted to change the frequency you would just change the parts with the *, you can use crontab.guru if you are unfamiliar with crontab setup.

share|improve this answer

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

share|improve this answer

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge

answered Aug 12 at 17:55

William YoungWilliam Young

2111 bronze badge