How to Read a File Line by Line Using Bash

1. Using the read Command in a While Loop
2. Conclusion
3. FAQ

Reading files line by line is a common task in scripting, especially when you want to process data efficiently. In Bash, this can be accomplished seamlessly using the read command. This command allows you to read a file one line at a time, making it easy to manipulate or analyze the content without loading the entire file into memory. Whether you’re working with logs, configuration files, or any text data, mastering this technique can significantly enhance your scripting skills.

In this article, we’ll explore different methods to read a file line by line using Bash. We will also provide clear examples and explanations to help you understand each approach. By the end of this guide, you’ll be equipped with the knowledge to handle file reading tasks effectively in your Bash scripts.

Using the read Command in a While Loop

Syntax

while IFS= read -r line
do
  echo "$line"
done < file_name

It reads the content of the file file_name one line at a time and prints the lines one by one in the terminal. The loop is executed until we reach the end of the file. The IFS is set to the null string, which helps to retain leading and trailing whitespaces.

Alternatively, the above command can also be replaced by the following command within a single line:

while IFS= read -r line; do echo $line; done < file_name

Example: Read the File Line by Line in Bash

In the example, we will read the file file.txt, which contains numbers in each line and then find the sum of all numbers in the file.

Content of file.txt

1
5
6
7
8
10

#!/bin/bash

sum=0
echo "The numbers in the file are:"
while IFS= read -r line
do
  echo "$line"
  sum=$(( $sum + $line ))
done < file.txt
echo "The sum of the numbers in the file is:$sum"

Output:

The numbers in the file are:
1
5
6
7
8
The sum of the numbers in the file is:27

It reads the numbers line by line from a file named file.txt and then sums up all those numbers and finally echoes the sum.

Example: Set Fields in Files to Variables

We can set fields in the file to variables by passing multiple variables to the read command, which will separate fields within a line based on the value of IFS.

Content of file.txt

Rohit-10
Harish-30
Manish-50
Kapil-10
Anish-20

#!/bin/bash

while IFS=- read -r name earnings
do
    echo "$name" has made earnings of "$earnings" pounds today!
done < file.txt

Output:

Rohit has made earnings of 10 pounds today!
Harish has made earnings of 30 pounds today!
Manish has made earnings of 50 pounds today!
Kapil has made earnings of 10 pounds today!

Here, each line in the file is divided into two segments as we have passed two variables to the read command. The first segment will be assigned to the name variable, which extends from the beginning of the line until the first -, and the remaining portion will be assigned to the earnings variable.

Alternative Methods to Read Files in Bash

#!/bin/bash

while IFS=- read -r name earnings
do
    echo "$name" has made earnings of "$earnings" pounds today!
done < <(cat file.txt )

Output:

Rohit has made earnings of 10 pounds today!
Harish has made earnings of 30 pounds today!
Manish has made earnings of 50 pounds today!
Kapil has made earnings of 10 pounds today!

Here, the filename file.txt is passed to the program as an output of the cat command.

Conclusion

Reading a file line by line in Bash is a fundamental skill that can greatly enhance your scripting capabilities. Whether you choose to use the read command in a while loop, a for loop, or the mapfile command, each method has its unique advantages. By mastering these techniques, you can efficiently process and manipulate text files in your scripts, making your workflows more effective and streamlined.

Now that you have a solid understanding of how to read files line by line using Bash, you can apply these techniques to various scripting tasks, whether it’s for automation, data processing, or system administration.

FAQ

1. How do I read a file line by line in Bash?
   You can use the read command within a while loop to read a file line by line in Bash.

2. Can I read a file without using the cat command?
   Yes, you can read a file directly using the read command or mapfile without needing cat.

3. What is the difference between using a while loop and a for loop?
   A while loop reads each line separately, preserving line breaks, while a for loop reads words, which may not maintain the structure of the file.

4. Is there a method to read all lines into an array?
   Yes, the mapfile command can be used to read all lines from a file into an array for easier manipulation.

5. Can I process files larger than memory using these methods?
   Yes, using the read command in a loop allows you to process large files without loading them entirely into memory.