Difference between revisions of "FITS"
Gary McGath (talk | contribs) (→FITS on Github: github change) |
Gary McGath (talk | contribs) (JHOVE performance) |
||
| Line 15: | Line 15: | ||
The HTML module in JHOVE is very slow and not very useful unless you're rejecting the 90% of the HTML files that aren't strictly valid. If you don't need it, edit xml/jhove/jhove.conf and remove the module element which refers to edu.harvard.hul.ois.jhove.module.HtmlModule . | The HTML module in JHOVE is very slow and not very useful unless you're rejecting the 90% of the HTML files that aren't strictly valid. If you don't need it, edit xml/jhove/jhove.conf and remove the module element which refers to edu.harvard.hul.ois.jhove.module.HtmlModule . | ||
| + | |||
| + | ==Improving JHOVE performance within FITS== | ||
| + | |||
| + | Spencer McEwen, who wrote most or all of FITS, tells me that JHOVE is the biggest bottleneck in FITS and that each time it's called, it grabs required schemas from the Web, instead of using local copies. I'll see if that's fixable. | ||
Revision as of 16:29, 16 November 2012
This page is for notes on how to optimize FITS.
Contents
Thread parallelism and memory consumption
FITS runs all its tools in parallel threads. This can result in heavy memory consumption, particularly if a big file is being processed. Harvard optimized FITS for DRS ingest, which can afford a lot of memory. In other environments this might cause thrashing.
One approach might be to add a command line or config parameter to limit the number of simultaneous threads.
FITS on Github
A fork of FITS is now up on Github. Let me know if you want to be added as a contributor. I've changed the repository name from fits to openfits to assuage concerns from Harvard.
The JHOVE 1.8 jars are now there.
Optimization tip
The HTML module in JHOVE is very slow and not very useful unless you're rejecting the 90% of the HTML files that aren't strictly valid. If you don't need it, edit xml/jhove/jhove.conf and remove the module element which refers to edu.harvard.hul.ois.jhove.module.HtmlModule .
Improving JHOVE performance within FITS
Spencer McEwen, who wrote most or all of FITS, tells me that JHOVE is the biggest bottleneck in FITS and that each time it's called, it grabs required schemas from the Web, instead of using local copies. I'll see if that's fixable.
