reading an ibm tape on unix
2007-12-25 9:04:00
We have a 9-track HP drive attached to one of our Sun Workstations running
SunOS 4.1.3. Using this we would like to read some IBM data files. We
are running into some problems and I would really appreciate if some one
can give us some hints for the following:
1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
problem is that it reads the entire data file as one record (40 megs!),
i.e., it does not put an end of record marker at the end of each record.
The file can not even be open using an editor on UNIX because it is so
large. I've tried using the unblock option too and it does not make any
difference. This problem is solvable by writing a C or Fortran program
that will read the tape and write an end of record marker to the output.
But it will be more convenient if we could use an UNIX command to read the
tape.
2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
embedded in a Fortran program through which we can make it known that it is
EBCDIC. Can any one suggest a way in which this data file can be read.
3) While one user is accessing the 9-track tape, how can we prevent other
users from using the mt or dd command to manipulate the tape drive. Does
it automatically know not to let more than one user access the tape drive?
***********************
Solution for question 1):
Gregor Fellers suggested using the following:
dd if=/dev/rmt/0 ibs=32704 | fold -448 > file1
This worked like a charm and saved me a lot of time and frustration. There
were a few other suggestions that may have also worked, but I did not try
it. I'll paste the repsonses at the end of my summary incase some one
wants to try other solutions.
Solution for question 2):
Use the dd command as above and include the option conv=ascii.
Solution for question 3):
If an user is using the tape drive, others will get a "device busy" message
when they also try to access it.
Thanks to all those who responded.
Vasantha
******************
Here are the responses:
Date: Tue, 6 Jun 1995 06:35:24 -0400
From: u2is9gef@gregsun.crrel.usace.army.mil (GREGOR E FELLERS )
Subject: Re: reading an ibm tape on unix
X-Sun-Charset: US-ASCII
I can help you with 1 & 3.
1. We routinely read tapes and have this problem, occassionally.
Rather than let dd write the output file I use fold, and this works
in every case that I have run across. Using your example:
dd if=/dev/rmt/0 ibs=32704 | fold -448 > file1
3. If a user is using the tape, other users will get a "device busy" or
"permissions denied" error (can't remember which). They will not
effect the actually tape operation.
*****************
From: Lyle Miller <lmiller@aspensys.com>
Subject: Re: reading an ibm tape on unix
To: Vasantha Narayanan <vnarayan@haverford.edu>
Content-Type: TEXT/PLAIN; charset=US-ASCII
Content-Length: 2785
All I know about this is that typically if multiple users try to
access a peripheral, the second user (and beyond) will see "device busy"
messages...and will not be able to access the device until it is freed.
****************
From: fhb@kestnbaum.com (Fred Berns)
Subject: Re: reading an ibm tape on unix
1)
You have specified a Conversion-Block-Size (cbs) but no conversion. If
you are not using the CONV= option, you don't need cbs. Nonetheless,
it sounds as though the file does not have embedded EOL
characters--which is not that unusual. You will have to handle it via
some other means.
2)
This is simple enough. Specify a CBS=22 and CONV=ASCII. This will
take care of the conversion for you.
******************
From: rhmoyer@mmm.com (Robert H. Moyer)
Subject: Re: reading an ibm tape on unix
Date: Tue, 6 Jun 1995 13:54:28 +0000 (GMT)
X-Mailer: ELM [version 2.4 PL23]
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
> Hi,
>
> We have a 9-track HP drive attached to one of our Sun Workstations running
> SunOS 4.1.3. Using this we would like to read some IBM data files. We
> are running into some problems and I would really appreciate if some one
> can give us some hints for the following:
>
> 1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
> 32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
> if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
> problem is that it reads the entire data file as one record (40 megs!),
> i.e., it does not put an end of record marker at the end of each record.
> The file can not even be open using an editor on UNIX because it is so
> large. I've tried using the unblock option too and it does not make any
> difference. This problem is solvable by writing a C or Fortran program
> that will read the tape and write an end of record marker to the output.
> But it will be more convenient if we could use an UNIX command to read the
> tape.
>
My quess is that the cbs option is not necessary here. Assuming that
you have 73 records per block with each block being 32704 in length then this
command should work for you (at least it always has for me in the past).
dd if=/dev/rst0 of=file1 bs=32704
Also, if you are reading the tape then it must be OK but it seems to me that
rst8 should be the correct device for a high density tape.
> 2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
> bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
> embedded in a Fortran program through which we can make it known that it is
> EBCDIC. Can any one suggest a way in which this data file can be read.
>
I don't know how to recognize the format of the data but you can use dd to
convert the data to ASCII like so.
dd if=/dev/rst0 of=file1 bs=33000 conv=ascii
> 3) While one user is accessing the 9-track tape, how can we prevent other
> users from using the mt or dd command to manipulate the tape drive. Does
> it automatically know not to let more than one user access the tape drive?
>
The second person will get a "Device Busy" type message when they attempt to
access the drive if someone else is using it at that time.
************************
From: Brad Burdick <bburdick@radio.com>
Subject: Re: reading an ibm tape on unix
Date: Tue, 6 Jun 1995 10:24:01 -0400 (EDT)
X-Mailer: ELM [version 2.4 PL21]
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 2241
>
> We have a 9-track HP drive attached to one of our Sun Workstations running
> SunOS 4.1.3. Using this we would like to read some IBM data files. We
> are running into some problems and I would really appreciate if some one
> can give us some hints for the following:
>
> 1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
> 32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
> if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
> problem is that it reads the entire data file as one record (40 megs!),
> i.e., it does not put an end of record marker at the end of each record.
> The file can not even be open using an editor on UNIX because it is so
> large. I've tried using the unblock option too and it does not make any
> difference. This problem is solvable by writing a C or Fortran program
Really? I use unblock all the time on some IBM ASCII data on tape with
no problems. For example, for input records of 80-byte fixed length with
a block size of 2000, this works just fine:
dd if=in of=out ibs=2000 cbs=80 conv=unblock
> that will read the tape and write an end of record marker to the output.
> But it will be more convenient if we could use an UNIX command to read the
> tape.
>
> 2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
> bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
> embedded in a Fortran program through which we can make it known that it is
> EBCDIC. Can any one suggest a way in which this data file can be read.
dd doesn't work on this one?
dd if=in of=out ibs=33000 cbs=22 conv=ascii,unblock
would work on normal EBCDIC tape data...
>
> 3) While one user is accessing the 9-track tape, how can we prevent other
> users from using the mt or dd command to manipulate the tape drive. Does
> it automatically know not to let more than one user access the tape drive?
The tape device should return a _busy_ status to all subsequent attempts while
the tape is in use.
**********************
From: Reto Lichtensteiger <rali@hri.com>
Subject: Re: reading an ibm tape
Date: Tue, 6 Jun 1995 10:23:07 -0400 (EDT)
X-Mailer: ELM [version 2.4 PL23]
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 1872
Back at the ranch, Vasantha Narayanan scribed:
: 1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
: 32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
: if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
: problem is that it reads the entire data file as one record (40 megs!),
Have you tried this:
% dd if=/dev/rst0 of=file1 conv=ascii ibs=32704 cbs=448
^^^^^^^^^^
'Cording to the man page on dd:
cbs is used only if ascii, unblock, ebcdic, ibm, or block
conversion is specified. In the first two cases, cbs char-
acters are placed into the conversion buffer, any specified
character mapping is done, trailing blanks trimmed and NEW-
LINE added before sending the line to the output.
: 2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
: bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
: embedded in a Fortran program through which we can make it known that it is
: EBCDIC. Can any one suggest a way in which this data file can be read.
You could have the fortran program call "dd" with -conv=ebcdic and output
to a pipe
: 3) While one user is accessing the 9-track tape, how can we prevent other
: users from using the mt or dd command to manipulate the tape drive. Does
: it automatically know not to let more than one user access the tape drive?
I don't think Sun's kernel implements locks in mt or dd, so you may be
out of luck here ... But, don't take my word for it; I don't have source
for SunOS ...<g> Call Sun support and ask them.
***********************
From: mattias@txc.com (Mattias Zhabinskiy)
Subject: Re: reading an ibm tape on unix
Hi Vasantha,
Have You tried to use command /usr/5bin/pax.
I'm not sure, if this command appropriate
in your case, but I know, that sometimes
it used, when dealing with tapes from
other POSIX compliant systems.
*********************
From: root@wisdom.maf.nasa.gov (Mark Hargrave)
Subject: Re: reading an ibm tape on unix
Vasantha,
Try the following command:
dd if=/dev/nrst0 of=<filename> cbs=80 conv=unblock
**********************
From: scowles@pangolin.Stanford.EDU (S. Cowles)
Subject: Re: reading an ibm tape on unix
X-Sun-Charset: US-ASCII
Ask archie for ansitape. This will read and write standard labelled
tapes on unix systems just fine.
******************
From: Dave Fetrow <fetrow@orac.biostat.washington.edu>
To: Vasantha Narayanan <vnarayan@haverford.edu>
Subject: Re: reading an ibm tape on unix
Content-Type: TEXT/PLAIN; charset=US-ASCII
On Mon, 5 Jun 1995, Vasantha Narayanan wrote:
> Hi,
>
> We have a 9-track HP drive attached to one of our Sun Workstations running
> SunOS 4.1.3. Using this we would like to read some IBM data files. We
> are running into some problems and I would really appreciate if some one
> can give us some hints for the following:
>
> 1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
> 32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
> if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
> problem is that it reads the entire data file as one record (40 megs!),
> i.e., it does not put an end of record marker at the end of each record.
> The file can not even be open using an editor on UNIX because it is so
> large. I've tried using the unblock option too and it does not make any
> difference. This problem is solvable by writing a C or Fortran program
> that will read the tape and write an end of record marker to the output.
> But it will be more convenient if we could use an UNIX command to read the
> tape.
Not the RIGHT solution but: sed was designed for this kind of problem. Perl
and maybe awk can deal with it as well.
3) While one user is accessing the 9-track tape, how can we
prevent other > users from using the mt or dd command to manipulate the
tape drive. Does > it automatically know not to let more than one user
access the tape drive? > > Thanks in advance. > > Vasantha > > > Vasantha
Narayanan > Academic Computing Center > Haverford College, PA. email:
The usual solution is to write a "checkout/checkin" routine; an SUID root
program that can change the ownership/permissions of a device that
shouldn't be shared.
*********************
From: perryh@pluto.rain.com (Perry Hutchison)
Subject: Re: reading an ibm tape on unix
> We have a 9-track HP drive attached to one of our Sun Workstations running
> SunOS 4.1.3. Using this we would like to read some IBM data files. We
> are running into some problems and I would really appreciate if some one
> can give us some hints for the following:
>
> 1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
> 32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
> if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
> problem is that it reads the entire data file as one record (40 megs!),
> i.e., it does not put an end of record marker at the end of each record.
Unix does not implement the notion of a "record" or "end of record
marker" at the OS level. A file is simply a sequence of bytes, and
it is entirely up to the application to assign meaning to them. However,
many applications (including most of the editors) adhere to the convention
that lines end with the ASCII linefeed character, 0x0a. What you need
is a program which will insert a linefeed after every 448 bytes, something
like this (untested):
------------------------ cut - cut - cut ------------------------
#include <stdio.h>
#define RECLEN 448
main()
{
int n;
char c[RECLEN+1];
while ((n = fread(c, 1, RECLEN, stdin)) > 0)
{
fwrite(c, 1, n, stdout);
putchar('\n');
if (n < RECLEN)
break;
}
}
------------------------ cut - cut - cut ------------------------
Now if the executable is named "addlf" you can use dd to handle
the tape's blocking requirements and pipe the output to addlf for
linefeed insertion:
dd if=/dev/rst0 ibs=32704 | addlf > file1
And yes, it would be nice if dd had this capability built in.
> The file can not even be open using an editor on UNIX because it is so
> large.
EMACS could probably handle it, but that is a whole other story (and
better written by someone who actually uses the beast).
> 2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
> bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
> embedded in a Fortran program through which we can make it known that it is
> EBCDIC. Can any one suggest a way in which this data file can be read.
The "conv" option to dd can handle the EBCDIC => ASCII translation.
> 3) While one user is accessing the 9-track tape, how can we prevent other
> users from using the mt or dd command to manipulate the tape drive. Does
> it automatically know not to let more than one user access the tape drive?
Only one process will be able to open the tape at any one time,
but there is not a stock mechanism to keep someone else from
fiddling between, say, an mt command which positions the tape
and an immediately-following dd which reads it. You need some
kind of volume manager, which would operate by assigning ownership
of the tape drive to a particular user and setting the permissions
to prevent access by others.
********************
From: mshon@sunrock.East.Sun.COM (Michael J. Shon {*Prof Services} Sun
Rochester)
Subject: Re: reading an ibm tape on unix
Cc: mshon@sunrock.East.Sun.COM
X-Sun-Charset: US-ASCII
|
|1) We have one file on 9 track with Logical Record Length = 448, BLKSIZE =
|32704, FB, ASCII, bpi=6500. I'm able to use the dd command (dd
|if=/dev/rst0 of=file1 ibs=32704 cbs=448) and read the data file. But the
|problem is that it reads the entire data file as one record (40 megs!),
|i.e., it does not put an end of record marker at the end of each record.
|The file can not even be open using an editor on UNIX because it is so
|large.
In Unix, there are no end of record markers in non-text files.
Text files use linefeed characters as a (sort of) record delimiter.
The only place unix uses real record markers is on tapes.
If the data is in text form, then you could use the "split" command
to break it up into smaller chunks, suitable for editing, but it has a limit
on how many separate files it can make.
A simple C program could break it up into many more files if you like.
If there is some change which you want to make over ALL of the data,
then 'sed' can process files of any size.
Once again - unix has *no concept* of record delimiters.
| I've tried using the unblock option too and it does not make any
|difference. This problem is solvable by writing a C or Fortran program
|that will read the tape and write an end of record marker to the output.
Sorry, no.
|But it will be more convenient if we could use an UNIX command to read the
|tape.
|
|2) We also have a data file that is EBCDIC with BS=33000, RL=22, BF=150
|bpi=1600, odd parity and unlabeled. Is there any UNIX command that can be
|embedded in a Fortran program through which we can make it known that it is
|EBCDIC. Can any one suggest a way in which this data file can be read.
The dd command can do ebcdic conversions : (from the dd man page) :
conv=ascii Convert EBCDIC to ASCII.
ebcdic Convert ASCII to EBCDIC. If converting fixed-
length ASCII records without new-lines, set up
a pipeline with dd conv=unblock beforehand.
ibm Slightly different map of ASCII to EBCDIC. For
fixed-length ASCII records without new-lines,
see above.
There are several flavors
of ebcdic, and it's translation may not be exactly what you need.
It should be close, however.
|
|3) While one user is accessing the 9-track tape, how can we prevent other
|users from using the mt or dd command to manipulate the tape drive. Does
|it automatically know not to let more than one user access the tape drive?
As long as a program has the tape device "open", no one else can use it,
(like while dd is reading the tape)
but in between operations, others could get in there.
SunOS has no built-in way to "reserve" the tape drive.
Comments
Got something to say?
You must be logged in to post a comment.
