How It Works

This page will be a moderately technical account of the technology behind CASA and the related InterMix software. There's a lot of written material about these inner workings, but it will take some time to assemble them in one place.

This page is currently little more than a placeholder, so that pages that must refer to how something works can go ahead and include the link to this one, without that being a dead link.   It seems to be easier to get finished pages up on the web by including all the links first, and making them point at first only to placeholders, rather than leaving the other pages very badly unfinished by the absence of those links.   We will leave this short explanation of the process up here on this page until enough material has been accumulated beneath the horizontal rule to remove the need for apology.

There's two topics here that should be separated. One is a discussion of the prototype software, its development, and what the process of improving it, while the other concerns the Internet and WWW interface -- the adaptation of the prototype software to make it online interactive software. Both topics will be discussed here, but this page will be mostly about the former. The ideas and programming efforts behind the prototype software are original and perhaps even unique, for which they deserve a more prominent treatment, while the attempts to get this up on the web are just more of what everybody is doing these days.

A new page called Putting it on the Web will discuss the details of HTML, Java, Javascript, CGI, Perl, PHP, and other tools and techniques that are befuddling millions at this very moment. That page will take something of a diary or journal-like approach, written mostly in the first person singular by Doug Wilson (and anyone else who wants to help). Meanwhile the rest of this page will deal with the core social technology and its prototype implementation.

For more general information about putting up the new web interface software, there is a new (and very incomplete) web interface analysis and design page .


Material actually about "How It Works" follows  ...

This is part of a series of messages or pages about the software to be made available on the SocialTechnology.ca website. Some of this software has already been written, some of it needs to be written, but most of it just needs minor adaptations from existing software either written by me or written by other people under the GPL or other free distribution licenses.

This collection of software is mostly aimed at social technology purposes, the facilitation of social activities, especially the choice of social relationships, but including much more such as almost all of what we now know as law and government.

But the software is for much more than that, and it includes programs for the reorganization of the Internet and programs for the development, maintenance, and distribution of other programs. The latter approach owes much to the insight of Roger Eaton, who recognized that the ideas described in my web pages would have an application in the open source software development community.

Open source software is one of the great success stories of the modern age, a truly remarkable phenomenon whereby software better than any you could buy for some purpose is available for free in the original programming language. Go to Source Forge for more information.

The CASA-like mechanisms for helping organize the Internet and especially facilitate the development of open source software are actually much easier to design, write, and explain than the CASA software itself. That shouldn't be too surprising -- CASA has to deal with human beings while the net and software stuff only has to deal with the net and the software on it.

(Material about early experiments with Javascript and other web tools that was here has been moved to the new Putting it on the Web page).

So, How Does It Work?

The prototype software consists of several components, all of which will be released as open-source software after some cleanup. They are mostly command-line programs with no user interface, which simply read in and process data files, creating new data files for processing by other programs.

As explained on the overall project summary page, the social network analysis and matching tools include many programs which are quite generic and are used as is by other components of the over-ambitious overall project. These common tools deal perform numerical analysis and combinatorial optimization processes on whatever data they are given. There were originally developed and tested with linguistic data as part of an entirely different component of the overall project.

More specifically social analysis software was developed for processing social survey data, and will be used for the very similar data to be collected from web users via the forms on these pages.

When users fill out a form, the resulting data is collected with data from as many other users as possible then put through numerical methods very much like the ones used by psychologists and sociologists throughout the 20th century. But where they stopped at that point and published their findings, we will use this data as an input to combinatorial optimization programs.

Combinatorial optimization is the process of improving the way things combine or link up. People are linked by the social relationships between them, some of which are rigidly formal like the relationship between employer and employee or husband and wife. Such well-defined connections can easily be represented as a collecton of small circles joined by lines, as in this illustration: (image file to be added here).

This is often referred to as a network, since a more regular version might look something like a fishing net, but in mathematical parlance it is called a graph. The branch of mathematics that deals with such graphs is called graph theory, and has nothing to do with bar charts, histograms, or anything a businessman or statistician would call a graph.

Typically the numerical analyst working with questionnaire data will work with vectors and matrices, which are lists and tables of numbers, respectively, while the graph theorist deals with lists of vertices (or points) and the edges (lines) joining them. However these two approaches are related and any graph (network) can be represented as an adjacency matrix in which the rows and columns of the matrix represent vertices. Where there is an edge connecting two vertices the corresponding rows and columns have a non-zero value, while vertices with no edge connecting them have a zero where the corresponding rows and columns intersect, as shown here: (image file to be added here).

A very common operation performed by numerical analysts on questionnaire data is called factor analysis or principal component analysis. It involves creating an autocorrelation or covariance matrix that shows how the answers given to the test questions are correlated, then the eigenvalues and eigenvectors of this matrix are extracted -- such terms are explained with examples (on a page not yet written). The adjacency matrix which describes a graph may in some cases also be treated as an autocorrelation matrix, and its eigenvectors found. That is very important for what follows. It allows us to do factor analysis on structures such as defined by a collection of web pages and the hyperlinks that join them.

(this page is very much under construction ... for some of the information that should follow right here, please see ... )


Copyright © 2000 Douglas P. Wilson



Copyright © 2009   Douglas Pardoe Wilson

Other relevant content:

New: Social Technology through Diagrams

New: Social Techs novel online

New: Social Technology Blog

New: Social Technology Wiki

Please see these web pages:

The main Social Technology page.

Find Compatibles , the key page, with the real solution to all other problems explained

Technological Fantasies , a page about future technology

Social Tech a page about Social Technology, technology for social purposes.  I think I was the first person to use this phrase on the Internet, quite a long time ago.


Roughly corresponding to these web pages are the following blogs :

Social Technology the main blog, hosted on this site, with posts imported from the following blogger.com blogs, which still exist and are useable.

Find Compatibles devoted to matching people with friends, lovers, jobs, places to live and so on, but doing so in ways that will actually work, using good math, good algorithms, good analysis.

Technological Fantasies devoted to future stuff, new ideas, things that might be invented or might happen, such as what is listed above and below.

Sex-Politics-Religion is a blog about these important topics, which I have been told should never be mentioned in polite conversation.  Alright that advice does seem a bit dated, but many people are still told not to bring up these subjects around the dinner table.

I believe I was the first person on the Internet to use the phrase Social Technology -- years before the Web existed.

Those were the good old days, when the number of people using the net exceeed the amount of content on it, so that it was easy to start a discussion about such an upopular topic.  Now things are different.  There are so many web pages that the chances of anyone finding this page are low, even with good search engines like Google.   Oh, well.

By Social Technology I mean the technology for organizing and maintaining human society.  The example I had most firmly in mind is the subject of  Find Compatibles , what I consider to be the key page, the one with the real solution to all other problems explained.

As I explained on my early mailing lists and later webpages, I find that social technology has hardly improved at all over the years.   We still use representative democracy, exactly the same as it was used in the 18th century.  By contrast, horse and buggy transporation has been replaced by automobiles and airplanes, enormous changes.

In the picture below you will see some 18th century technology, such as the ox-plow in the middle of the picture.  How things have changed since then in agricultural technology.  But we still use chance encounters, engagements and marriages to organize our home life and the raising of children.  

I claim that great advances in social technology are not only possible but inevitable.  I have written three novels about this, one preposterously long, 5000 pages, another merely very very long, 1500 pages.  The third is short enough at 340 pages to be published some day.  Maybe.  The topic is still not interesting to most people.   I will excerpt small parts of these novels on the web sometime, maybe even post the raw text for the larger two.


This site includes many pages dating from 1997 to 2008 which are quite out of date.  They are included here partly to show the development of these ideas and partly to cover things the newer pages do not.  There will be broken links where these pages referenced external sites.  I've tried to fix up or maiintain all internal links, but some will probably have been missed.   One may wish to look at an earlier version of this page , rather longer, and at an overview of most parts of what can be called a bigger project.

Type in this address to e-mail me.  The image is interesting.  See Status of Social Technology

Copyright © 2007, 2008, 2009, Douglas Pardoe Wilson

I have used a series of e-mail address over the years, each of which eventually became out of date because of a change of Internet services or became almost useless because of spam.  Eventually I stuck with a Yahoo address, but my inbox still fills up with spam and their spam filter still removes messages I wanted to see.  So I have switched to a new e-mail service.  Web spiders should not be able to find it, since it is hidden in a jpeg picture.   I have also made it difficult to reach me.  The picture is not a clickable link.  To send me e-mail you must want to do so badly enough to type this address in.  That is a nuisance, for which I do apologize, but I just don't want a lot of mail from people who do not care about what I have to say.


Cross-References:

Easy Introduction

Progress Report -- (old)

Putting It Up on the Web

Science and Technology

Web Interface -- Analysis and Design


Copyright © 2009   Douglas Pardoe Wilson