[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Limits of grep?
On Tue, 26 Sep 2000, Shachar Shemesh wrote:
I suggest using xargs with such a large number of files. Assume you're in
the directory with those 10000+ files:
find . -name "*"|xargs grep -s -l pattern
This will return a list of files that include the pattern, and will
supress unwanted errors. Then, you could feed the output of this to a
further `|' (pipe), and do more analysis.
--Ariel
> Short explanation regarding the use of wildcards in Unix commands:
>
> Unlike what you may or may not know from Dos, the one responsible for the
> wildcard expansion is not the command itself - you will not find in grep
> instructions regarding what to do with filenames having asterixes. Instead, it
> is the shell that does the wildcard expansion, and passes that as a list of
> files to grep.
>
> You may have already noticed, and if not, you may notice in the future, that
> almost all basic shell commands accept first all parameters, and only last the
> file or files to work on:
> chmod <params> file
> ls <params> file
> etc.
>
> The reason for that is precisely so that you can use this shell feature to
> perform wildcard expansion.
>
> The down side of this is that the limit you have encountered is a shell limit,
> not a grep limit. The only thing I can suggest to you is to use a program that
> enumerates the files, and performs grep on each one. Luckily, such a program
> exists. It is called "find".
>
> The basic command line to do what you want is:
> find . -name * -exec grep pattern {} ;
>
> The above will not work as typed, because the shell will perform all sorts of
> manipulations on the command line. What you have to type is
> find . -name \* -exec grep pattern \{\} \;
>
> You may want to tweak the grep parameters a little to get the results exactly
> the way you want them.
>
> Shachar
>
>
> Subba Rao wrote:
>
> > Hi
> >
> > I have a directory of 10000+ text files and would like to search for
> > some strings in these files. When I tried using "grep" command with an
> > asterisk, I get the error message somthing to the effect,
> >
> > "File argument list too long"
> >
> > What is the file argument limit for grep? I guess you need the grep source
> > for this. I did not find any information in the man page.
> >
> > Are there any other recommended tools to search through such large list of
> > files?
> >
> > Thank you in advance.
> >
> > Subba Rao
> > subb3@attglobal.net
> > http://pws.prserv.net/truemax/
> >
> >=================================================================
> > To unsubscribe, send mail to linux-il-request@linux.org.il with
> > the word "unsubscribe" in the message body, e.g., run the command
> > echo unsubscribe | mail linux-il-request@linux.org.il
>
>
> =================================================================
> To unsubscribe, send mail to linux-il-request@linux.org.il with
> the word "unsubscribe" in the message body, e.g., run the command
> echo unsubscribe | mail linux-il-request@linux.org.il
>
--
Ariel Biener
e-mail: ariel@post.tau.ac.il Work phone: 03-6406086
fingerprint = 07 D1 E5 3E EF 6D E5 82 0B E9 21 D4 3C 7D 8B BC
=================================================================
To unsubscribe, send mail to linux-il-request@linux.org.il with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail linux-il-request@linux.org.il