Disk Pak for Unix
2007-12-25 8:28:00
I did receive some replies on disk fragmentation.
----- Begin Included Message -----
>From adam%bwnmr4@harvard.harvard.edu Tue Sep 7 11:50:33 1993
Return-Path: <adam%bwnmr4@harvard.harvard.edu>
From: adam%bwnmr4@harvard.harvard.edu (Adam Shostack)
Subject: Re: Disk Pak for Unix
To: twhitely@tr1072.to.ford.com
Date: Tue, 7 Sep 93 11:55:47 EDT
Content-Length: 1162
You wrote:
| Hello Sun Guru's
|
| I am currently looking at a third party software package called Disk
| Pak for Unix. It is a disk defragmenter. It will show you how you
| disk is set up and also the arrangement of the disk as
| far as files, inodes and free disk space with any fragmentation present. My biggest concern
| with this package is that this package will break down the disk and reorder the disk. This
| brings up a question of liability. Has anyone used Disk_Pak? If so, how do you favor this package?
| Please email me all criticisms and concerns to twhitely@tr1072.to.ford.com. If you are also interested
| in some info on this package, send me email.
I'd use dump/restore if you really feel your disks need
degfragmenting. But, if you look at the way inodes work,
fragmentation is not nearly so big a problem on unix as under dos.
Adam
--
Adam Shostack adam@bwnmr4.harvard.edu
Systems Manager 617-732-7692
Surgical Planning Lab, Dept of Radiology Fax 732-7963
Brigham and Womens Hospital, Boston
----- End Included Message -----
----- Begin Included Message -----
>From @srlns1.srl.ford.com,@vm.cnuce.cnr.it:ACHILLM@IMIHSRA.BITNET Wed Sep 8 03:36:58 1993
Return-Path: <@srlns1.srl.ford.com,@vm.cnuce.cnr.it:ACHILLM@IMIHSRA.BITNET>
Date: Wed, 08 Sep 93 09:31:29 GMT
From: Martin Achilli <ACHILLM%IMIHSRA.BITNET@vm.cnuce.cnr.it>
Subject: Re: Disk Pak for Unix
To: Ted Whitely <twhitely@tr1072.to.ford.com>
Content-Length: 1152
Hello,
up to now,what I do is to (yearly) back up and restore all filesystems complete
ly. I do a double backup for safety. And before restoring, I rebuild filesystem
s on the disks.
This operation improves large file access performance (we work with Magnetic Re
sonance and Nuclear Medicine data: ave file size 1 Mb).
I suppose the program is like the DOS compress utility, so presumably you must
have at least one back up of your disks data.
Please send me more info, although I expect a program like this costs about 500
dollars (for one host ?), so perhaps, if you don't have too many disks, it is
cheaper to do what I do once a year.. :-)
Martin
-------------------------------------------------------------------------
Martin Achilli
Consiglio Nazionale Delle Ricerche * Tel: +392/26433648
Ist. di Neuroscienze e Bioimmagini * Fax: +392/26415202
c/o Medicina Nucleare * Email: ACHILLM at HSR.IT (Internet)
Ospedale San Raffaele * at IMIHSRA (Bitnet)
Via Olgettina 60 *
20132 MILANO (Italy) *
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
----- End Included Message -----
----- Begin Included Message -----
>From dsnmkey@guinness.ericsson.se Wed Sep 8 04:32:33 1993
Return-Path: <dsnmkey@guinness.ericsson.se>
Date: Wed, 8 Sep 93 10:31:49 +0200
From: dsnmkey@guinness.ericsson.se (Martin Kelly)
To: twhitely@tr1072.to.ford.com
Subject: Re: Disk Pak for Unix
Content-Length: 1560
Hello,
> If you are also interested in some info on this package, send me email
I'm interested. Can you semd me an info pack ?
> This brings up a question of liability.
It is most likely that the Disk Pak utility has no 'liability' if you
lose data or software. You are expected to provide your own backups.
You can minimize the risk of data loss by dumping the disk to a
free filesystem or disk (rather than tape), running the Disk Pak
program the way you want to, and then run a diff on all the files
and directories (this may take a while). Then you will know that
the Pak'd disk contains the exact same data as the backup and the
original disk.
If you meant to ask about reliability, then I do not know how reliable
it is. I guess you will have to ask the vendors.
However, the question arises if sufficient gains will be made even
after Pak'ing your disk. This is very much data set dependent and
application dependent. Your mileage may vary .....
Best Regards,
Martin.
============================================================================
Martin Kelly Tel: +31 1612 29358
Unix Analyst/Specialist Fax: +31 1612 29071
Ericsson Data Services Nederland, BV. (DSN)
PO Box 209
5120 AE Rijen MEMO: ERI.DSN.DSNMKEY
The Netherlands E-Mail: dsnmkey@guinness.ericsson.se
DSN - Providers of Unix and Networking Expertise to the Corporation
============================================================================
----- End Included Message -----
----- Begin Included Message -----
>From bobr@houston.nam.SLB.COM Wed Sep 8 09:18:05 1993
Return-Path: <bobr@houston.nam.SLB.COM>
Date: Wed, 8 Sep 93 08:17:37 CDT
From: bobr@houston.nam.SLB.COM ( Bob Reardon )
To: twhitely@tr1072.to.ford.com
Subject: Re: Disk Pak for Unix
Content-Length: 4654
Here is the reply from a previous question about fragmentation.
Not exactly what you asked about but may be useful anyway..
========================================================================
>From sun-managers-relay@ra.mcs.anl.gov Fri Jun 11 21:04:17 1993
Return-Path: <sun-managers-relay@ra.mcs.anl.gov>
Received: from ra.mcs.anl.gov by houston.nam.SLB.COM (4.1/SMI-4.9)
id AA11374; Fri, 11 Jun 93 21:04:14 CDT
Received: by ra.mcs.anl.gov id AA19287
(5.65c/IDA-1.4.4 for sun-managers-outbound); Fri, 11 Jun 1993 11:40:58 -0500
Sender: sun-managers-relay@ra.mcs.anl.gov
Received: from delta.eecs.nwu.edu by ra.mcs.anl.gov with SMTP id AA19281
(5.65c/IDA-1.4.4 for <sun-managers@ra.mcs.anl.gov>); Fri, 11 Jun 1993 11:40:52 -0500
Precedence: bulk
Received: from cgi.com (cggate.cgi.com) by delta.eecs.nwu.edu with SMTP id AA04484
(5.65c/IDA-1.4.4 for <sun-managers@eecs.nwu.edu>); Fri, 11 Jun 1993 11:40:33 -0500
Date: Fri, 11 Jun 1993 12:43:19 -0400 (EDT)
From: HILL@cgi.com (Bear)
Reply-To: HILL@cgi.com (Bear)
Followup-To: junk
Message-Id: <930611124319.240020f5@CGIVA>
Subject: Fragmentation Summary
To: sun-managers@eecs.nwu.edu
X-Vmsmail-To: SMTP%"sun-managers@eecs.nwu.edu"
Content-Length: 3412
X-Lines: 94
Status: RO
First off..Thanks to all who responded. You're all such a knowledgable crowd.
Also my apologies to those I may have missed. In summary:
1. Dump the disk to tape, newfs the filesystem, and restore the tapes.
2. Try to tune with tunefs or mkfs parameters
3. Unix Central catalog lists a product called Eagles Software Disk_Pak
(800)532-1771 (913)823-7257
4. Spend your time instead by placing often-used common filesystems in the center
of the disk, minimizing the seek time to go to the user filesystems at either
end.
5. You could try a fsck while unmounted...this will show %fragmentation.
6. And the most interesting is the following:
>From: SMTP%"stern@sunne.east.sun.com" 11-JUN-1993 11:11:31.26
To: HILL@cgi.com
CC:
Subj: Re: Anyone know of good defragmentation software?
Date: Fri, 11 Jun 93 11:07:40 EDT
>From: stern@sunne.east.sun.com (Hal Stern - NE Area Systems Engineer)
Message-Id: <9306111507.AA14455@sunne.East.Sun.COM>
To: HILL@cgi.com
Subject: Re: Anyone know of good defragmentation software?
be careful about "fragmentation". there is no internal
fragmentation in the UNIX filesystem -- files only
contain fragments in the last data block, never in
the middle of a file. so your file will comprise
full data blocks for most/all of its extent.
where you run into problems is with block placement,
and "working set distribution". the optimal block
placement strategy in BSD unix (FFS) tries to drop
the blocks down so that you minimize rotational delay.
the sunos 4.1.x UFS+ puts the blocks down in "clusters"
of 56k, making longer contiguous stretches that give
you much improved performance with SCSI disks (or other
disks that move a fairly large buffer in one shot).
the block placement can run into problems with a full
disk, and end up putting blocks where you have to
seek/rotate quite a bit to read the whole file. the
file isn't exactly fragmented, but it is suboptimally
laid out.
finally, there's the issue of what files you're using
and how they are arranged on the disk. if you can
keep a whole working set together, again you minimize
seek times.
what can you do?
(a) dump and restore the filesystem. this palces all blocks
optimally
(b) use something like nfswatch to track what files are
used most frequently. then run restore with
a list of files, and restore them in "favored" order.
this lets you keep frequently used files together
to cut down on seek times.
--hal
This summary was made possible from Knowledge grants from the following Contributors.
SMTP%"jdavis@cs.arizona.edu"
SMTP%"fetrow@biostat.washington.edu"
SMTP%"Birger.Wathne@vest.sdata.no"
SMTP%"smc@goshawk.lanl.gov"
SMTP%"leclerc@clamart.est.slb.com"
SMTP%"lsil!mhost!lonfs01!allen@fernwood.mpk.ca.us"
SMTP%"barnes@sde.mdso.vf.ge.com"
SMTP%"schmelin@cad.gmeds.com"
SMTP%"jeff@access.digex.net"
SMTP%"lem@cgi.com"
SMTP%"poc@cgi.com"
SMTP%"lemke@mitl.research.panasonic.com"
SMTP%"rwolf@dretor.dciem.dnd.ca"
SMTP%"bert@penril.com"
SMTP%"long-morrow@cs.yale.edu"
SMTP%"stern@sunne.east.sun.com"
SMTP%"exudnw@exu.ericsson.se"
SMTP%"morrow@cns.ucalgary.ca"
Thanks,
-Bear
--------------------------------------------------------------------------
Scott A. Hill (Bear)
Senior Hardware/Network Engineer
Carnegie Group Inc.
5 PPG Place
Pittsburgh, PA 15222
(412)-642-6900
--------------------------------------------------------------------------
----- End Included Message -----
----- Begin Included Message -----
>From jeff@access.digex.net Wed Sep 8 16:26:20 1993
Return-Path: <jeff@access.digex.net>
Date: Wed, 8 Sep 1993 16:25:48 -0400
From: Jeff Mallory <jeff@access.digex.net>
To: twhitely@tr1072.to.ford.com
Content-Length: 2907
I got a catalog from UnixCentral listing this product and read its claims. Piqued my
interest `cause it seemed to try to draw parallels between DOS-style file cluster
usage and Unix inode/cylinder group usage. Having just read a (most excellent) Hal
Stern article about unix file systems, I said they cannot do what they are claiming
because the wrong they say they right doesn't exist in UFS. So, I called UnixCentral
and asked for more product info, then faxed them parts of Hals` article that
basically rebutted the whole idea of unix file fragmentation. A salesperson called
me and said they had passed on the info to Eagle (the makers of Disk Pak) and they
were writing up a response (this was ~2 months ago.....).
So, I sent a note to Hal laying out this interchange and he replied that SunWorld (I
think) had approached him about reviewing Disk Pak and he had told them then what I
had sent to UnixCentral about unix "fragmentation" of disks.
(sounds of rummaging through 'net notes), A ha! It was actually in a sunmgr reply that
Hal talked about defragging:
>From: SMTP%"stern@sunne.east.sun.com" 11-JUN-1993 11:11:31.26
To: HILL@cgi.com
CC:
Subj: Re: Anyone know of good defragmentation software?
Date: Fri, 11 Jun 93 11:07:40 EDT
>From: stern@sunne.east.sun.com (Hal Stern - NE Area Systems Engineer)
Message-Id: <9306111507.AA14455@sunne.East.Sun.COM>
To: HILL@cgi.com
Subject: Re: Anyone know of good defragmentation software?
be careful about "fragmentation". there is no internal
fragmentation in the UNIX filesystem -- files only
contain fragments in the last data block, never in
the middle of a file. so your file will comprise
full data blocks for most/all of its extent.
where you run into problems is with block placement,
and "working set distribution". the optimal block
placement strategy in BSD unix (FFS) tries to drop
the blocks down so that you minimize rotational delay.
the sunos 4.1.x UFS+ puts the blocks down in "clusters"
of 56k, making longer contiguous stretches that give
you much improved performance with SCSI disks (or other
disks that move a fairly large buffer in one shot).
the block placement can run into problems with a full
disk, and end up putting blocks where you have to
seek/rotate quite a bit to read the whole file. the
file isn't exactly fragmented, but it is suboptimally
laid out.
finally, there's the issue of what files you're using
and how they are arranged on the disk. if you can
keep a whole working set together, again you minimize
seek times.
what can you do?
(a) dump and restore the filesystem. this palces all blocks
optimally
(b) use something like nfswatch to track what files are
used most frequently. then run restore with
a list of files, and restore them in "favored" order.
this lets you keep frequently used files together
to cut down on seek times.
--hal
Jeff Mallory
jeff@millie.loc.gov
----- End Included Message -----
----- Begin Included Message -----
>From ups!uniq.com.au!glenn@warrane.connect.com.au Wed Sep 8 21:23:46 1993
Return-Path: <ups!uniq.com.au!glenn@warrane.connect.com.au>
Date: Thu, 9 Sep 93 08:11:26 EST
From: glenn@uniq.com.au (Glenn Satchell - Uniq Professional Services)
To: twhitely@tr1072.to.ford.com
Subject: Re: Disk Pak for Unix
Content-Length: 2831
With the SunOS' BSD Fast File System disk fragmentation is not such a
big deal any more. Normally I just do a full backup/newfs/restore about
once per year to defragment my disks.
regards,
--
Glenn Satchell glenn@uniq.com.au | "This is a unix system.
Uniq Professional Services Pty Ltd ACN 056 279 335 | I can do this easy."
PO Box 70, Paddington, NSW 2021, (Sydney) Australia |
Phone 02 360 7434 Pager 016 287 000 Fax 02 331 2572 | - Lex, Jurassic Park
"Sun Accredited System Consultants" |
----- End Included Message -----
I don't believe that a Sun filesystem fragments constantly. Most replies quote that defragmenting
was done once a year. This is good for users that have a very small network, but for my case, it
can be a nightmare cause I have a very large network with over 550 nodes! We also have over 30 servers
containing more than 4gig's of disk space. Disk_Pak for unix will let you defragment disks actively or
interactively. By using disk pak interactively, defragementing can be done overnight or over the weekend.
Whether or not the software is reliable, every good administrator knows to keep up to date backups of
everything in the event of lost data. Disk_Pak also has an X interface that allows you to scan the disk
to see how badly fragmented it is. A large network which might require a network license, the cost would be
about $3000. For those who would like more info or prices on Disk_Pak for unix, here is the following information:
Eagle Software (Disk_pak for Unix)
123 Indiana Avenue
P.O. Box 16
Salina, Kansas 67402-0016
Phone (913) 823-7257
Fax (913) 823-6185
Contact: Darin Boyer, Products Representative
PS. Thanks for everyone's responses.
#############################################################################
# #
# _____ ______/ ___________/ _____________ _/ _/ #
# _/ _/ __ / _/ _/ #
# _/ ___________/ __ _/ / _/ _/ _/ #
# _/ _/ __ / _/ _/ _/ #
# __/ ___________/ ______________/ _/ _/ __/ _/ #
# #
# Ted Whitely Unix Tech. Support #
# System Administrator Ford Motor Company #
# Truck Operations Danou Technical Center #
# TSD Systems Group Suite 6000 #
# Internet: twhitely@tr1072.to.ford.com 16630 Southfield Rd. #
# Phone: (313) 390-5259 Allen Park, MI 48101 #
# FAX #: (313) 845-7501 U.S.A. #
#############################################################################
Comments
Got something to say?
You must be logged in to post a comment.
