|
| Home | Crystal | jmol | jPOWD | Chem | X Ray | Dana | Strunz | Properties | A to Z | Images | Share | News | Help | About |
How is the Mineral Website created?
|
The mineral website is a hobby that I do when I am not out doing consulting for petroleum geology. In answer to your questions I'll explain how I used the programming tools to create the mineral website. The actual mineral database is in Microsoft Access. This is where I input the basic data. To produce the mineral website, I use MS access queries, forms and/or reports for exporting the mineral data out of access in a text format that is usable by SAS or for importation into a MySQL table. I have different access reports, forms, and queries for most of the tables in the mineral database. The next step is done using the SAS language. For example, I use SAS to create each of the individual mineral pages using their macro language to format each page as an individual file (all 4,714+ of them). The SAS programming takes care of generating the links, creating the load files for MySQL, and creating the other pages. SAS creates the alphabetical pages, Strunz classification pages, Dana classification pages, and the chemistry pages. When all the pages are created, I use Expression Web ver 2 to chase down broken links, provide web continuity, and supply the final formatting. 09/15/12
10/01/11Due to high server load, the webmineral website has been moved to a new Virtual Dedicated Server (VDS) and no longer resides on a shared hosting site. The virtual dedicated server (cloud server) has many advantages over a physical server and none of the disadvantages. My thanks to the fine folks at Webmineral's hosting site, Portal Web hosting who helped make the changeover as easy as possible. Webmineral users should notice a faster response time while using the site. I have been going over the site looking for problems and have fixed a few. If there are any I missed, I would appreciate users letting me know (dbarthelmy@webmineral.com) of any problems because the site is so large and I can't review every page. 12/31/09Added a highly experimental feature to Webmineral, a Pivot™ interface using a subset
of mineral data and mineral images.
Users can download the software and install it using one the following
installation keys. 11/07/09Converted the alphabetical listing pages over to 100% php code. Now the alphabetical listing is always updated with the current new data. The old style short-cut icons still available with the addition of an icon for crystal structure files that are viewable using the jPOWD structure applet.
10/05/09Added 600 missing mineral images from various sources to the image photo library. The mineral images index page now has the capability to display individual copyright information in thumbnail gallery view. A paging function has been added to avoid having excessively large pages load when viewing images. These changes are listed as follows:
8/15/09Updated the jPOWD mineral structure files with all current CIF data
from the American
Mineralogist Crystal Structure Database. The data reflect current
mineral nomenclature changes. There are now 11,220 mineral structure
files in Webmineral. Thanks to Bob Downs of the
RRUFF project for providing this
information. I feel that valid mineral names should not be changed to reflect some IMA subcommittee's idea of mineral groupings or to make their Excel files sort correctly. Discrediting a mineral name is certainly a valid IMA activity. Renaming valid mineral names violates scientific rules of precedence in the literature and the practice should be abandoned.Here is a 10 year summary of Webmineral.com's monthly traffic for the home page from the start-up of the domain name on 7/15/2000. The older stats from the http://web.wt.net/~daba/Mineral site (1996 - 2000) are not included. The yearly low points are the summer months of June, July, and August.  8/05/09Major update of the site with new minerals to 7/09. The mineral count in webmineral.com is now up to 4,714 after updating all minerals to June of 2009. Several minerals were renamed due to IMA decree. Alas, fluorapatite is no more (renamed to apatite-CaF). I have added internal redirection links for all old names so that http://webmineral.com/ data/ Fluorapatite.shtml will automatically bring up the new mineral name. There have been a number of modifications and additions to the website which need mentioning. These are: All Dana classification numbers have been modified by converting all single digit elements that make up the Dana ID by prepending a "0" to them. For example, 1.3.15.4 is now 01.03.15.04. This makes sorting easier rather than using custom sort routines as was needed for the old numbers. The Stunz classification numbers have been updated to the latest Nickel-Strunz (version 10) classification. These numbers are courtesy of James A. Ferraiolo. Jim's scholarship on mineral classification goes back to 1982 when he published A Systematic Classification of Nonsilicate Minerals that updated the Dana (7th Edition) numbers. Randy Ernst, Editor of the Canadian Micro Mineral Association (CMMA) has provided webmineral with updated mineral fluorescence data. These have been incorporated into the species pages and are a welcome addition over the previous fluorescence data. All cumulative corrections received from 2005 to 2009 via e-mails from users have been applied in this update. Many of these were for chemical compositon. The chemical compositions are easily derived from the empirical formula or mineral formula. I spent some time during this update cycle correcting chemical composition by using the formula charge balance to flag suspect entries in the access database that feeds webmineral. 3/28/09Converted the site from FrontPage format to CSS. This required the conversion of all FrontPage tags to their CSS counterparts. When Webmineral was first created in 1997, the average browser resolution was S-VGA (600x800), access was 98% phone modem, and the most common browsers were NCSA Mosaic and Netscape 2.0. These older browsers will no longer work in Webmineral because they don't support tables. I will continue to use a minimal amount of html code that can be viewed quickly and easily in most common browsers. This site has been evolving from a simple html site to one that uses
server side scripting (php) and now relies on CSS.
If you experience undesirable problems with the new layout, please let me know ( ). Sections of the site will be upgraded to the new format in phases so the old website format may appear in some parts of webmineral. 3/08/09Enhanced the Table of X-Ray spacing to allow searching on D1, D2, D3, and chemical formula in any order. The search is based on reported values from each mineral where a diffraction file has been reported. Two theta values can be calculated for different wavelengths based on commonly used X-Ray anode values. In addition, the user can input any value for X-Ray wavelength and the 2 theta angles are calculated for that wavelength. Optional Search Query - To Reset, Click Here Some important observations: 8/31/08Upgraded OpenAds from version 2.4.3 to Openx 2.6.1. This
upgrade fixes security flaws in the
Some of the technical specifications on statistics and reporting in Openx include:
5/31/08News Section. Added paging and search functionality to the news section on webmineral as the section was starting to get too large for a simple listing. The news section contains e-mails from webmineral's advertisers outlining advanced sales, new acquisitions, and other news items. Included with the advertiser information newsletters and e-mails are informational e-mails from various geological and mineralogical professional organizations. I have also, from time to time, added links to mineral show reports on the web.
1/26/08Upgraded OpenAds
This OpenAds software upgrade was intended to be added last October, 2007. Webmineral uses a development server for testing upgrades and major code changes. The previous development server was based on Red Hat version 8.0 which has been running for the last 5 years. The server quit running during the attempted upgrade and no amount of tinkering with the apache server (v 1.33) could get it running again. While Red Hat makes a good distribution of the LINUX kernel, their user support is lacking. Without a development server to test the upgrades, I needed to configure
a new LINUX box. My research lead to the Ubuntu LINUX distribution as a
suitable replacement for the development server. To make a very long story
short, I installed Ubuntu on a spare PC and I have been slowly getting the
web server configured to match the production server that currently hosts
webmineral.com. Ubuntu's development community
Best of all, both OpenAds and Ubuntu are free. 7/22/07Paper crystal models. Steffen Weber has created a replacement version of JCrystal called KrystalShaper that can produce paper models of the example crystals on the crystallography section of Webmineral. Paper models for the 32 crystal classes have been reproduced in .pdf format. These models can be printed on heavy card stock to construct a physical model. Please note: the paper model forms may be simplified from the JCrystal forms to avoid overly-complicated cut-outs. Students may want to try and construct a physical model of the crystal forms for illustration purposes. These models will help teach symmetry operations needed to identify crystal forms. The student is encouraged to mark on the paper models noting mirror planes and rotational axes. The hemimorphic forms (in bold type below) can be identified by the [0 0 -1] face in the appropriate crystal class.
2/18/07Upgraded phpAdsNew from version 2.0.4 to version 2.0.8. This upgrade should help advertisers view statistics since the phpAdsNew MySQL tables were repaired and optimized. 10/15/06Major new modification to Webmineral. View interactive crystal structure diagrams using jPOWD Applet. In March, webmineral added crystal structure diagrams using jmol. All attempts to install the structure diagrams on the species pages failed because jmolApplet.jar is large (600 kb) which resulted in unacceptable download times on webmineral's development server. I looked for alternatives on the web to display crystal structure diagrams. This is the first major website to use the jpowdjpx.jar applet which comes with jPOWD from Materials Data, Inc. The program generates x-ray pattern and structure simulation by reading crystallographic information files (.cif) and generates a simulated x-ray diffraction pattern from the atomic positions. The output of binary crystal files (*.jpx) from the jPOWD program and the jpowdjpx applet allows web display of crystal structures. 9/25/06
7/04/06New addition to webmineral.com. See Webmineral News and view items of interest from the mineral world. The topical items may include news of upcoming auctions and mineral specimen sales from webmineral's advertisers, mineral show reports from other sources on the web.
5/29/06Upgraded phpAdsNew from version 2.0.4 to version 2.0.8. This upgrade should help advertisers view statistics since the phpAdsNew MySQL tables were repaired and optimized. 3/15/06Major new addition to Webmineral. View interactive crystal structure diagrams using Jmol Applet. This capability just recently came available with the publication of the open-source program, Jmol, written in java and the companion java applet for web use. This program replaces Chime and does not require that the user install a "helper" plugin. The ability to visualize and manipulate crystal structures completes the capabilities of webmineral as a mineral database. Users can also download and run Jmol as a stand-alone application. The generation of these mineral structure required the following:
6/27/05I am nearly complete in the upgrade of the banner ad rotation software
on webmineral.com from the existing perl code to a new php/MySQL system
from phpAdsNew. The planned change over
to the new banner ad system is July 1st. This will provide users with a
better selection of links should the user desire the opportunity to purchase
mineral specimens from the best of the mineral dealers who sell them on
the internet.
6/07/05I have got the site mostly restored to the pre June 1st configuration
after they were able to chown the ownership from root to webmineral. I have
re-configured the webmineral's development server to be in line with the
purenetlistings.com site set-up. 6/01/05On June 1st, my provider,
www.vitalstream.com
handed the site over to one of their business associates,
www.purenetlistings.com.
All this without prior warning. 5/07/05The mineral count in webmineral is now up to 4,442 after updating all minerals to 2004. The crystallography section has been converted to a php/MySQL database for easier updates in the future. All minerals in the database now have a publication date (See IMA Status on species pages). This has allowed an accurate assessment of the number of non-discredited, valid minerals (1,645) present prior to the establishment of the International Mineralogical Association (IMA) and the New Minerals and Mineral Names (CNMMN) in 1959. Chemical composition pages for each element have been changed to include elements which are present in the empirical formula but are not present in the chemical formula. The changes are illustrated by the following excerpt from the chemical analysis pages (Titanium) accessed by the periodic table.
Komarovite example shows the presence of titanium in the chemical formula and Ti is highlighted in red. Mindat.org, Webmineral.com and MinMax websites are cross-linked by thousands
of URL references. Although people who use software programs to download
whole websites appear to be a minority, when they do, it affects all of
our users. Especially if they have a broadband connection to the web.
If you really need to have an off-line copy of webmineral, just purchase
the
PHOTOGRAPHIC GUIDE
TO MINERAL SPECIES 2nd Edition which features the
data files from webmineral with thousands of mineral images.
If your address is found in the above list and you wish to have the block removed, then send me an e-mail promising not to download the site and I will remove the block. 11/07/04Added a new class of calculated data for each mineral species, the Fermion Index and complimentary Boson Index. There are some unusual properties which are linked to these values. These properties are not yet well understood. It has been speculated that these properties may link metaphysical (spirit) and physical (real) worlds. This is pure speculation at the moment. 4/07/04Jeffrey G. Weissman, co-author of the Photographic Guide to Mineral Species is currently updating his Photographic Guide with new and larger images. Jeff has given webmineral permission to use some of his older mineral images from the current version of the CDROM. These new mineral images are in-addition-to the 1,475 images already in webmineral's image files assembled from individuals over the last three years. This now makes webmineral the most comprehensive (as measured by number of separate species) picture index site on the web. Please note: The 1,105 new species added from the Photographic Guide represent some of the rarest and hardest-to-find minerals in the world. These images are the "Good, the Bad, and the Ugly" and are an invaluable reference to anyone who needs mineral data. 3/04/04There is a whole class of information which can be derived from the data contained in webmineral. Calculated values for various different parameters have been added to the species pages in a section titled "Calculated Properties" along with the more familiar "Physical Properties" and "Optical Properties". The new calculated variables are:
1/24/04This upgrade of webmineral.com converted all the mineral pictures to a MySQL/PHP database and reporting system. The previous set-up used static pages and server side includes (SSI). This was not the correct way to handle the image data in a large website. The conversion to php to handle image libraries should help site maintenance considerably. The look-and-feel of the old pages has been preserved with the added benefit that many more images can be easily added in the future. Future plans are to convert as much of webmineral's content to an on-line database as possible. This will help the update cycle and allow more up-to-date content rather than the batch process done on a periodic basis.8/28/03The mineral count in webmineral is now up to 4,339. This update includes all the error corrections and additional information that have accumulated in the Access database over the year. The chemical analysis have been error checked by converting the chemical compositions to oxides and comparing the sums of oxides to the original analysis. The most important change with this version of webmineral is the use of cascading style sheets to format certain aspects of mineral data. Please note that the rotational inversion axes of 3 (3BAR), 4 (4BAR), and 6 (6BAR) and overbar hkl values may not be visible to older browsers (Netscape v4 and Internet Explorer v5) which don't use cascading style sheets (.css). This means that 15% of webmineral's users cannot view these changes.
8/11/03Due to technical problems with webmineral's provider, vitalstream.com, the site was unavailable from Saturday, August 9, till Monday morning, August 11. We are examining the problem to prevent a reoccurrence. Thank you for your patience. 8/01/03New Feature. The index page has been modified to show random User Tips and Factoids. This list is summarized in this link. If you want to add to this list, please e-mail suggestions to the webmaster. 7/05/03Preparations for a major site update with new and revised data is underway. As part of the update, mineral reflectivity data for hundreds of opaque species were added. Webmineral now has the largest collection of spectral opaque mineral reflectivity data on the web. This reflectivity data can be accessed along with the other data types on the mineral species pages (See Reflectivity). This data is also summarized in the determinative mineralogy tables for opaque minerals. New Feature. Need a fast and easy mineral label for your specimens? Just visit the species page for the particular mineral you need a label for and print a handy copy to place under your specimen. The label (already filled in with necessary data) is included in the "See Also" section of the species page. For more options in creating mineral specimen labels, you can take the time and download Ososoft's Mineral Label 5.0 for free. Bob's Rockshop is a mirror site so you can also download it there. 10/05/02The mineral count in webmineral is now up to 4,281 after adding 61 new minerals and deleting 35 obsolete entries. This update includes all the error corrections and additional information that have accumulated in the Access database over the year. The big news is webmineral now has a working version of Gladstone-Dale Compatibility Index programmed in SAS and the data is now available on the species pages. Many thanks to Dr. Anthony Kampf of the Los Angeles Museum of Natural History who provided the background information and a copy of his ChemComp program which does the calculation. The reference for this technique is Mandarino, J. (1981) paper in Can, Min, (1981), Vol 19, pp 441-450). Basically, the compatibility index (CI) compares the Gladstone-Dale constants (KC) for the chemical analysis using the oxide values for each mineral and (KP) derived from density and index of refraction values. The CI is a measure of the "goodness" of the data based on physical and chemical properties for each mineral. An additional benefit is that the index of refraction can be calculated (from KP) if the optical data are not present. The application of this technique pointed out deficiencies in the mineral data. The results are as follows:
These have been corrected and work is underway to correct some of the deficiencies in the data when the oxide percentages are derived from the empirical formulas (∑ ± 100% ). The tolerance is 95% to 105% of the sum of oxides. 7/21/02Webmineral has obtained a copy of Dana's 7th edition. All those wonderful crystal drawings missing in modern mineralogy references are now in the process of being added to webmineral using Jcrystal. It is a slow process so I can only do a few at a time. In addition, I have also purchased a copy of "Determinative Mineralogy and Blowpipe Analysis" by Brush & Penfield, 1906. This book, first published in 1874, is the state-of-the-art manual for qualitative blowpipe analysis of chemical elements in minerals. I am adapting some of the element tables (see Borax and Phosphate) and the analytical techniques and have been re-doing portions of the chemical element pages. 9/10/01This update now has the chemical composition of all minerals in place. The elements are calculated as % and as oxide % where applicable. This update also includes the new minerals for 2000 (4,255 total). The main changes are as follows:
4/10/01I am trying out a new machine translation service from InterTran to replace the Babelfish service that I was using. It provides a larger selection of languages and the Java applets work after the translation. If there is enough of a demand for this service, I could upgrade the service from a free translation to a paid-basis translation. The paid-basis translation would allow the selection of more appropriate grammer for the Mineralogy Database and would run on a dedicated server for faster translations. 3/20/01I am moving the Mineralogy Database over to a new server. My old ISP wanted to charge me an arm and a leg to host the mineralogy database site because of its size. The temporary URL is http://webmineral-com.temp-url.com/default.shtml until the domain name is transferred. The main change is from a NT server to a UNIX server. The old server was just too overloaded with site bandwidth running at 22 Gb/month and the cgi scripts were corrupting the counter files on a regular basis. Plus, microsoft does not recommend running cgi scripts on a NT domain server. I intalled a new search engine on the webmineral.com site. For a preview, see ../search.shtml. It is a shareware cgi perl script system from fluid dynamics that allows administration from web browser pages and it seems to work very very good. It is much better than Microsoft's search engine supplied with FrontPage. I have submitted the domain change to registrar.com. When the changes propagate though the DNS system, webmineral.com should be active on the new servers at http://www.publichost.com. Hopefully, the changes should be transparent to the users of the Mineralogy Database. If there are any problems, please notify dbarthelmy@webmineral.com 3/14/01There are file corruption problems with the cgi programs on the Mineralogy Database's server. Please bear with me until I change the site over to a server that can handle the load. 1/18/01Changed the "See Also:" section on the mineral pages. Marc Farve of Alkali-Nuts and Herb Yeates of Franklin Minerals have changed their access to mineral data and the reference URL's have changed. The "see also" section of the mineralogy database is created from a listing and I can change it at any time easily because I use SSI includes for the data. I have also expanded the search engine links in the "see also" section to include most of the currently available search engines on the web. This is really handy because it allows specific data to be called up by just clicking on the link. 9/11/00Added a redesigned advertisers page to allow clients of the mineralogy database to log-on and review their usage statistics in real time. There are more images being added to the site with photos generated by the site's sponsors. These photos are exceptional in quallity and may be viewed on the specimen page in the database. The images are heavily cross-link to the data pages and vice versa. I am adding more images on a regular basis since the advertising revenue allows for the increase in the size of the site. 8/11/00The site has gone through a major update. One of the biggest changes is the inclusion of .shtml pages for the individual mineral pages. Unfortunately, most sites that link to the individual mineral species page are now out-of-date. This means those sites who have linked http://web.wt.net/~daba/data/Quartz.html or ../data/Quartz.html will now need to link to ../data/Quartz.shtml. Lets go on with the additions:
7/15/00The data pages on the old site (http://web.wt.net/~daba/Mineral) have been deleted. Please use ../ as the URL for the Mineralogy Database. I have also added advertising banners to help support the increased costs associated with this move. So if you like this site, please visit the sponsors who are kind enough to place their ads in the Mineralogy Database. I hope I have left enough redirection pages and .htaccess files on the old site so all those people who link to the old site can find the new site. If you have any questions, complaint, gripes, and etc. about this move please let me know (dbarthelmy@webmineral.com). 5/20/00I am over my quota of disk space on the wt.net personal account at http://web.wt.net~/daba/Mineral. I guess this means I will have to go commercial (http://www.webmineral.com) with the database. Please bear with me until I can transfer to the new domain. The internal layout of the database will remain the same, only the domain name will change I am going to keep all the main pages on the old site and link to the new site when I make these changes. The pages I will keep in the old site are http://web.wt.net/~daba/Mineral/index.htm http://web.wt.net/~daba/Mineral/chemical.html http://web.wt.net/~daba/Mineral/X-Ray.html http://web.wt.net/~daba/Mineral/crystall.html http://web.wt.net/~daba/Mineral/danaclass.html http://web.wt.net/~daba/Mineral/strunz.html http://web.wt.net/~daba/Mineral/determin.html http://web.wt.net/~daba/Mineral/specimens.html http://web.wt.net/~daba/Mineral/Alphabetical_Listing.html http://web.wt.net/~daba/Mineral/other.html http://web.wt.net/~daba/Mineral/dana/newdanasort.html This will avoid having the site go dead for people who just link those pages. If you have any questions or concerns, please feel free to ask. 1/14/00Added a search engine feature from FreeFind. I have not tried it out extensively, so have at it and report back your impressions. The mineralogy database site was too large for their spider to index fully so there may be some holes in this feature. It will not do element searches for "Ca Fe SO4" but it will find minerals where the chemical analysis are present (not present on all minerals in the database). For example, try a search on "sodium iron monoclinic". It does a good job of finding all the minerals with "Canada" in the locality section. Try "JCrystal" and you will get all the pages with Steffen Weber's JCrystalApplet on them. 10/20/99Steffen Weber offered the use of his JAVA based, JCrystalApplet for the webdisplay of crystal shapes (see the help file for a description). This program allows a user to manipulate a crystal shape with mouse and keyboard control with a realism almost equal to actually holding a real crystal model. It is such a nice program that I added the JCrystal version to the crystallography table for interactive examples of crystal morphology for the different crystal systems. I left the original static images in place so that older browsers without Java support could view the content. Steffen also offered me the use of a beta copy of the actual stand-alone software, JCrystal, which creates the html files that JCrystalApplet uses. Using this program, I have started adding crystal morphology images to some of the pages (e.g., quartz, benitoite) for the more common minerals as an added feature to the mineral database. If there are any people out there who use this program to create crystal shapes for the different minerals, I would appreciate a copy of the hkl files so I could add that mineral to the ones already in the database. 7/21/99This is a major update involving every page of the Mineral Database. Because of the many items that were changed, I decided to list those in a table.
6/1/99Revised the crystallography table to include examples of crystal morphology for the different crystal systems. This fixes one of the weakest links (content wise) in the mineralogy database. I have also placed a prototype of a new layout of the mineral pages (see example) for testing on different browsers. When it is field tested, then I'll update the whole site with new entries in the database. 5/14/99Added a statistical web tracker to monitor the usage on the mineralogy database. This tracker does not keep individual statistics, just general information. I was kind of interested in knowing who was using the database, who is linking to it, and how many new users were accessing the information. It is open to public access by clicking on the ExtEME tracking logo on the bottom of the index page. 12/2/98Revised the links page to include more sources of mineral data on the web or from commercial sources. 8/3/98Added Dana's New classification. This arrangement of mineral species is based on Dana's New Mineralogy by Richard V. Gaines, H. Catherine Skinner, Eugene E. Foord, Brian Mason, and Abraham Rosenzweig, (ISBN: 047119310-0) Copyright © 1997, John Wiley & Sons, Inc. This material is used by permission of John Wiley & Sons, Inc. Thanks to Wiley's for granting permission. Abraham Rosenzweig graciously sent me some unpublished material dealing with the silicate classification. He writes, "A detailed explanation of the logic behind the development of the class numbers in this group was to have been part of the new book, and manuscript for this was furnished to the publisher. However, the publisher chose to eliminate this section, along with some other matters, in order to save pages, as the book was already large." I have incorporated this data in the Dana classification pages. 4/3/97Took pictures of my personal mineral specimens and added a page of images. I used a 200mm telephoto lens with macro focusing to take the pictures and scanned them after the photo lab was finished processing them. The images were enhanced using Adobe PhotoShop 4.0 and resized to 640 x 480 resolution. 5/29/96Created the precursor program for the Mineralogy Database using the data files from Microsoft Access to create a Win95 Hypertext Help file using RoboHelp95. |
The majority of the pages in this website use plain old html with SSI and
php. This way the site can be updated in sections. The replacement product
for MicroSoft's FrontPage is Expressions Web. The newest version of Expressions
Web is Expressions Web 2. They even added scripting support for the open
source php software in the product.
