Getting Sum, Count and Distinct Count of a file

Question

Hi all this is a UNIX question.
I have a large flat file with millions of records.
col1|col2|col3
1|a|b
2|c|d
3|e|f
3|g|h
footer****
I am supposed to calculate the sum of col1 =9, count of col1 =4, and distinct count of col1 =c3
I would like it if you avoid external commands like AWK. Also, can we do the same by creating a function?
Please bear in mind that the file is huge
Thanks in advance

Answer 1

This sounds like homework for a shell class, but here goes. Save into a file, maybe "doit". Run it like this:
$ ./doit < data
<snip>
#!/bin/sh
got0=0
got1=0
got2=0
got3=0
got4=0
got5=0
got6=0
got7=0
got8=0
got9=0
sum=0
cnt=0
IFS='|'
while read c1 c2 c3 junk; do
# Sum and count
echo "c1=${c1}"
case "${c1}" in
[0-9] )
     sum=$(expr ${sum} + ${c1})
     cnt=$(expr ${cnt} + 1)
esac
# Distinct
case "${c1}" in
0 )     got0=1;;
1 )     got1=1;;
2 )     got2=1;;
3 )     got3=1;;
4 )     got4=1;;
5 )     got5=1;;
6 )     got6=1;;
7 )     got7=1;;
8 )     got8=1;;
9 )     got9=1;;
esac
done
echo "cnt=${cnt}"
echo "sum=${sum}"
echo "distinct="$(expr $got0 + $got1 + $got2 + $got3 + $got4 + $got5 + $got6 + $got7 + $got8 + $got9)
<snip>