SVDdiagnostic

Purpose

The purpose of SVDdiagnostic is to diagnose Rietveld refinements for singularity or quasi singularity of the matrix of normal equations in order to propose a cure.

Background about singular value decomposition (SVD)

For a simple introduction, see: Kalman, D. (1996) A Singularly Valuable Decomposition: The SVD of a Matrix, College Math Journal, 27:1, 2-23. In spite of its recent (in terms of mathematical breakthroughs) introduction [Golub, G.H. and Kahan, W. (1965). Calculating the singular values and pseudo-inverse of a matrix, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal. 2, pp. 205�224.], SVD is today the method of choice for solving linear algebra and especially least squares problems for three main reasons.

• Its formulation is very general and applicable to rectangular matrices. Eigenvalues and eigenvectors of square matrices for example appear as particular cases of SVD formulation.
• A numerically stable and computationally inexpensive algorithm exists for SVD decomposition.
• Singular systems are handled with the same ease as regular systems, decomposing them into a product of three matrices, each of them straightforward to invert or adapt in case of singularity or quasi singularity.

The method is so general and powerful that it would be able to find and handle properly at the least squares level e.g. the symmetry constraints on anisotropic thermal motion and the constraints on the displacement of atoms at Wyckoff positions. This could be performed in the same fully general and automated way for a limited cost in computing time without running into matrix inversion problems, and without programming any symmetry-specific knowledge into the algorithm. Its only drawback is that its numerically stable and computationally economical decomposition algorithm is an implicit algorithm, making it very difficult to understand. In contrast, checking the correctness of the result of the decomposition is straightforward. Users should then treat this algorithm as a black box, as many users presumably already do for LU or Cholesky decompositions that are instead straightforward to understand algorithms.

The SVD method has numerous uses outside crystallography, ranging from sociological and epidemiolological studies, marketing, finance, image compression, compressed storage of scalable characters or pictograms etc.. However, as noticed in e.g. Watkin, D. (1994). Acta Cryst. A50, 411-437, the SVD method is not in much practical use in crystallography, but that is certainly not because it would not find applications. We propose in Mercier, Le Page, Whitfield and Mitchell (2006). Submitted to J. Appl. Cryst. an application of SVD to the diagnostic and cure of ill conditioned Rietveld refinements. In this application, SVD would be difficult to bypass. Until about 1990, crystallographic least-squares refinement was very much focussed on economy of computer time and computer memory. By that time the refinement aspects of inorganic and small-molecule crystallographic packages were already pretty much cast into concrete. Had the timing been different, especially inorganic-oriented crystallographic packages could have benefited from the use of SVD-based least-squares refinement because it is much more flexible and informative than more traditional inversion methods.

Hands-on introduction to SVDdiagnostic

Users interested in the concepts behind SVDdiagnostic should consult Mercier, Le Page, Whitfield and Mitchell (2006). Submitted to J. Appl. Cryst., which should become available soon. Users interested only in the know-how can go by with the following PowerPoint tutorial. If Microsoft Office is available on your computer with PowerPoint installed, you will be able to view the Tutorials by just clicking on the hyperlinks below. If PowerPoint is not installed, installing a free PowerPoint viewer from the Microsoft website will achieve the same result.

Tutorial on setup and running of SVDdiagnostic

Tutorial on objective diagnostics and cures with SVDdiagnostic

Tutorial on pre-conditioning the normal matrix

If you are a developer or a user of a crystallographic least squares package currently not included
If you would like SVDdiagnostic to process the output of your preferred package as well, please send as an email attachment a sample unedited complete output (with no parts cut off or comments added) of a typical least squares run including printout of the matrix of normal equations to: yvon.le_page@nrc.ca after double-checking that it is not in the process of being added. You will be requested to perform a few tests on the upgraded load module of SVDdiagnostic, after which the new load module will be made available to all for free.