A Bloggy Christmas Letter

Posted on by
2

It’s been a year of blogging badly so I might as well end it with a Christmas letter. I won’t excuse my absence from these bloggy pages; I have repeatedly warned my readers that I blow hot and cold. I am either all in or all out. For most of 2016 blogging has been off the agenda.

In April, I traded my secure well-paying job in St. Louis Missouri for a risky venture in Santa Fe New Mexico. It was a crazy move. The money was about the same, but everything else, benefits, long-term prospects, working conditions, and career opportunities were worse. I didn’t care. I was coming up on six years in St. Louis: my longest stretch in any job, and I was looking for any opportunity to move.

I’ve changed jobs a lot over my long so-called career. I’ve left voluntarily, I’ve been downsized, and I’ve been fired. Being fired is the most dramatic and educational of the three. I was fortunate to be fired from my first “serious” job. I was young, dumb, fearful, and easily taken advantage of. Getting fired fixed all that. You don’t understand the working world until you’ve been fired. I’d recommend it for everyone. Getting fired provides lifetime immunity to one of mankind’s most crippling conditions: fear of change. I don’t fear change; I seek it out. Moving to New Mexico was a change for change’s sake. The only thing unusual about this move was a fixed destination.

In the past, I went where the money was. This time my wife was largely responsible for the location. She was tired of apartment living in St. Louis. We considered buying a house in the area but we didn’t view St. Louis as “home.” I wanted to live somewhere in the affordable mountain west and she wanted somewhere warm. Santa Fe was warm, mountainous, and, if my new job went well, just barely affordable. So we sold off our bags, packed up my photography gear, and moved west.

Things did not work out. My new job imploded a week before closing on the house we were building. For months we had watched the construction and we were just about to move in when the deal collapsed. I almost enjoyed the predicament. On my birthday, I found myself, unemployed, homeless and uninsured, but at least I had my health! Never undervalue your health.

Resting on a petrified stump on the trail. Old farts require more rest stops on the way up. I am so glad that trail running was not a thing when I was young. I didn’t see any trail runners in Yellowstone today. The presence of bears, wolves and mountain lions, all of which can run trails a lot faster than millennial showoffs, puts the brakes on such behavior.

Resting on a petrified stump on the trail. Old farts require more rest stops on the way up. I am so glad that trail running was not a thing when I was young. I didn’t see any trail runners in Yellowstone today. The presence of bears, wolves and mountain lions, all of which can run trails a lot faster than millennial showoffs, puts the brakes on such behavior.

We scrambled and quickly moved to Bozeman Montana to stay with my father. Since my mother’s death three years ago he has spent his summers in Bozeman alone. He was glad to have some company and we were glad to have a place to stay while I resumed the chore of looking for another job. I didn’t mind our summer in Bozeman. We hiked a lot, visited Yellowstone a few times, drove up to Glacier National Park, visited Missoula and Butte, and took lots of pictures. I also spent many hours scanning, restoring, and annotating old family slides. I found some real gems like this shot of my mother as a young teenager with baby “Tommy.”

Evelyn, my mother as a teenager, holding baby Tommy. I have no idea who baby Tommy is but I am pleased with the restoration of this old slide of Hazel’s. I am generally dissatisfied with most of my restoration work but every now and then you come close to the image in your head.

Evelyn, my mother as a teenager, holding baby Tommy. I have no idea who baby Tommy is but I am pleased with the restoration of this old slide of Hazel’s. I am generally dissatisfied with most of my restoration work but every now and then you come close to the image in your head.

It took me a few months to find another job because I refused to consider contracting or benefit free employers. Programming jobs are plentiful but programming jobs with stable employers that offer good benefits are not as plentiful as they used to be. Programming’s halcyon days are over. Modern trends are all negative and I fear that Trump will not make programming great again. In thirty years AI systems will replace all but the most brilliant and creative of programmers. Corporate IT drones, like moi, will join the dinosaurs.

Fortunately, being an old fart, I no longer care about long-term trends. I will be comfortably dead before global warming melts the ice caps and drowns coastal cities. I will also be retired and parasitically feasting on a plethora of millennial and gen-X funded social security programs when AIs decimate the ranks of ordinary programmers. My only concerns are short term. So I was pleased to find a good, benefit endowed, job in Meridian Idaho. Meridian is a growing suburb of Boise. Meridian is not as attractive as Santa Fe but the beauty of Idaho’s landscape matches New Mexico’s. Idaho is actually more mountainous than New Mexico and housing is more affordable. The funds we were about to plow into a New Mexico house will instead buy a larger fraction of a larger house in Idaho. It’s not the mountain state we aimed for but the skiing is better, (it’s a winter freaking wonderland outside as I write), plus the Pacific coast and Yellowstone are both within an easy drive. It will do for now.

In the coming year, I will strive to blog more often. I see many stark raving diatribes in your future, but as Hillary voters just painfully discovered,1 the worm does not always turn as expected.

  1. It was tempting to use the word “learned” but deplorables of the left have no need of learning; they already know it all?↩

Falling Colors Technology a BHSD Crony that needs Competition

Posted on by
Reply

BHSD (Behavioral Health Services Division) is a New Mexico state agency that doles out federal and state funds to a variety of small, ostensibly health related, programs. For example, in the state of New Mexico, BHSD runs a program called Synar1 that attempts to cut down on merchants selling cigarettes to minors. One Falling Colors employee characterized the program as “mostly stick and no carrots.” Synar funds a stable of ambush inspectors that descend on merchants hoping to catch them selling to minors. It’s a standard bit of well-intentioned government coercion. If you are wondering what’s in it for the merchants stop wondering: it’s all stick. They lose sales and face fines. If you are wondering why the state of New Mexico supports Synar follow the money. Depending on the dubious statistics compiled by Synar administrators the state could lose millions of dollars of federal grants if the percentage of offending merchants exceeds an arbitrary threshold. Synar, in the twisted minds of state bureaucrats, “generates revenue.”

In addition to saving the state from the unconscionable scourge of teenage smoking BHSD also funds a mishmash of programs to prevent drug addiction, help the mentally ill, subsidize methadone treatments and reimburse psychologists, psychiatrists and other health professionals for counseling and other services. BHSD’s budget for all these operations is, according to Mindy Hale, roughly fifty million dollars per year. In the greater wasteful schemes of government this is small beans, even for New Mexico, but, it’s still fifty million public dollars so it’s not out-of-bounds to ask, what are the taxpayers getting for their money?

If you are naïve enough to think the intended clients of BHSD’s largesse, the teenage smokers, the mentally ill, and the drug addicts, garner the lion’s share of that fifty million dollars you’re probably a statist or a moron, but I repeat myself. Many years ago a wise old wag, when badgered about the high cost of landing a man on the moon, chirped, “None of that money was spent on the moon!” While some of BHSD’s fifty million is directed to clients, the moon, the lion’s share goes to contractors, service providers, and BHSD internals. Whether the state of New Mexico and the federal government are getting good value for their money is debatable; what’s not debatable is that some IT service providers are doing very well from themselves.

Two IT providers consume a significant share of BHSD IT funds: Optum New Mexico and Falling Colors Technology. The founders of Falling Colors, Mindy Hale and Pamela Koster3, claim Optum bills the state of New Mexico roughly four million dollars per year for the onerous job of cutting checks. It’s important to understand that Optum is not dispensing their own money. They are simply managing a pool of funds that are replenished by state and federal tax dollars. Yes, it takes money to manage money. You have to pay auditors, comptrollers, and other financial professionals to make sure the funds are not redirected into questionable pockets. Surely you don’t think New Mexico’s corruption free government would abscond with unwatched dollars?

fctlogo

The Falling Colors Technology Logo. This logo was designed by a competent graphic designer. I’ve observed an inverse relationship between the quality of company logos and the products and services they offer.

Still, four million seems a bit steep for providing a routine service that any experienced financial entity like a bank could do, and to the state’s credit, they have recognized this and are in the process of renegotiating Optum’s four million dollar fee. Optum has responded with a “this isn’t worth the damn hassle” attitude. If they cannot get their four million they’re threatening to pull out of the state and cede the check cutting business to others. How much of this is hardball negotiating, corporate whining, or even the truth, is hard to determine. The only thing that seems certain is that there is a business opportunity for an IT provider if Optum makes good on their threat and leaves New Mexico.

Falling Colors Technology, a little company that is already extracting about one million dollars per year from the state, is angling to take over Optum’s fund disbursement role. In standard insider crony fashion, they hope to keep this transfer quiet and elude potential competitors. Why go through all that messy inefficient public bidding? There’s only one problem with their business plan. Falling Colors has absolutely no experience managing funds. There is nobody on their staff that could be considered a financial professional. They are planning to hire staff, but I have to wonder why BHSD, and the state of New Mexico, are considering flushing millions of dollars through an entity that has no financial expertise and has already received a formal letter of warning for shoddy IT work.

Instead of branching out into lines of business that they have no experience with Falling Colors efforts would be better invested in fixing their core problems and they have lots of core problems. Let’s look at what nearly one million dollars or public funds per year buys from Falling Colors Technology.

Your one million is buying a few unreliable, crash prone, insecure, low volume websites geared towards BHSD staff and service providers. When I first ran the following SQL query on the database that backs many Falling Colors websites I was alarmed at the results.

SELECT  iq.WeekNumber ,
        AVG(iq.ErrorCount) AS AvgWeekErrors ,
        MIN(iq.ErrorCount) MinWeekErrors ,
        MAX(iq.ErrorCount) AS MaxWeekErrors ,
        STDEV(iq.ErrorCount) AS StdDevWeekErrors
FROM    ( SELECT    CAST(CONVERT(VARCHAR(8), TimeUtc, 112) AS INTEGER) AS DayNumber ,
                    COUNT(1) AS ErrorCount ,
                    MIN(DATEPART(iso_week, TimeUtc)) AS WeekNumber
          FROM      dbo.ELMAH_Error
          GROUP BY  CAST(CONVERT(VARCHAR(8), TimeUtc, 112) AS INTEGER)
        ) iq
GROUP BY iq.WeekNumber

Falling Colors websites were crashing about twenty times per day. On some days the crash count exceeded fifty. I thought to myself, “If this doesn’t dramatically improve this little company is doomed.” I’ve worked with lots of bug infested software over my long career but twenty to fifty crashes per day, distributed over a few dozen users, was an entirely new level of unreliability.

Why is it so bad? The developers at Falling Colors, like developers everywhere, bitched about “inherited code.” Basically, this means they’re working with code that they didn’t entirely write themselves. Developers complaining about inherited code is so common that software managers rightly label it whining. Software developers bitching about inherited code is like civil engineers griping about inherited bridges. The world is not created fresh every day. The inherited code base is a source of problems but the main reason Falling Colors exhibits such a high crash rate is simply a lack of formal quality control.

Testing at Falling Colors is mostly performed by one beleaguered Business Analyst. She runs through a series of basic web page checks after significant new releases. This is a very low standard of testing for modern software development. Falling Colors does not practice many common quality control techniques. For example, most development environments support a variety of internal testing tools. Falling Colors is a Visual Studio shop and Visual Studio has built-in unit testing tools and supports a host of third-party add-ons. Developers focused on quality, spend as much time implementing internal units test as they do writing production code. There is an entire coding regime known as TDD that strongly promotes writing tests before you write software to pass the tests. At the end of June 2016, there were precisely zero internal unit tests in Falling Color’s code base. In addition to missing internal unit tests, there were no repeatable or scripted tests, no large case tests, and no stress tests. Lack of formal testing combined with misplaced developer optimism is a recipe for high error rates and Falling Colors is really boiling that pot.

Buggy insecure low volume websites are a dime a dozen. There’s a lot of crap out there. If Falling Colors cranked out standard public websites we would click on and ignore their rubbish. Unfortunately, being intertwined with BHSD, the users of Falling Colors websites do not have the option of clicking on. Making things worse, Falling Colors hosts a substantial amount of HIPAA protected information.

HIPAA is a set of federal guidelines that outline how health providers and their contractors must protect information that might be used to uniquely identify people. HIPAA penalties, for both providers and individuals, are severe if protected information is either accidentally or willfully disclosed. You can go to jail for exposing HIPAA protected information.

HIPAA guidelines list common data elements that must be protected. There is only one way to properly protect these elements: full element encryption. Every single data element should be encrypted and the keys should be rigorously guarded by a small number of individuals. Even developers, especially developers, should never see the unencrypted information. This is the way things should work, but, if you have followed the news about an unending stream of website hacks and data breaches, you’re probably aware that this is not how it works in the big nasty world.

It’s certainly not the way things are working at Falling Colors. With the exception of website passwords, which were only hashed in the last year,4 HIPAA data is stored in plain, ready to hack, text. If I were an IT savvy methadone user in the state of New Mexico I would be reluctant to disclose personal information to CareLink, TreatFirst, Prevention, or any of the Falling Colors managed programs. One HIPAA breach and your methadone habit is on Facebook.

Falling Colors is fully cognizant of their shabby security and are planning to eventually fix it. They’re taking steps to harden their websites and tighten up their loose databases but they are not, as of the end of June 2016, pursuing a full element encryption regime. Anything short of full element encryption is just putting lipstick on the security pig. Currently, Falling Colors is a HIPAA breach in waiting. BHSD would be well advised to insist on an immediate and independent full security audit of Falling Colors systems!

BHSD should also demand a fair and public RFP (Request for Proposal) process when seeking IT contracting services. Currently, some individuals in BHSD, in connivance with Falling Colors, are delicately crafting RFPs that are designed to exclude Falling Colors competitors. This is a blatant abuse of the public RFP process and the perpetrators should be ashamed of themselves. Crony state contracting may be business as usual in New Mexico but it is not in the interests of the pubic, BHSD, or even Falling Colors. Cronies without competition invariably turn into parasites and BHSD, which recently suffered a bedbug outbreak in their Santa Fe offices, has enough of those.

  1. The Synar program is named after Congressman Mike Synar of Oklahoma. How many tax dollars would be saved if it was illegal to name things after politicians?↩
  2. The founders of Falling Colors are questionable sources; their claims should be subjected to a high standard of scrutiny.↩
  3. Yes, incredibly user passwords were stored as plain text for years. This is monumentally inept.↩

Milliblog: Photo Captions

Posted on by
Reply

This blog is still alive and kicking. I post when I post. Currently, my energies are deployed on other fronts. If you absolutely must get your Analyze the Data not the Drivel fix look over my extensive millibloggy photo captions.  Here’s a typical example:

The “miraculous staircase” in the Loretto Chapel. After paying your three dollar entrance fee you can enter the chapel and see the staircase. When I was there people were milling around while a recorded message told the breathless story of the miraculous staircase. The story goes something like this. The chapel needed a staircase. Some nuns prayed to the sky fairy of carpenters and low and behold a carpenter showed up with a bag of simple tools. Over the next month this remarkably skilled carpenter fashioned this beautiful wood staircase using only his simple tools. Apparently he constructed the staircase from the floor to the upper level without central supports. Even now the staircase lacks the standard central beam of spiral staircases. The masses were amazed! How could the staircase stand without a central support? Surely this is the work of the divine. This is what passes for a miracle among sky fairy believers. It’s the same type of magical thinking that invokes aliens to explain the pyramids. Just maybe the carpenter knew what he was doing and had the technique to pull it off. Occam’s razor people: It cuts deeply.

The “miraculous staircase” in Santa Fe’s Loretto Chapel. After paying your three dollar entrance fee you can enter the chapel and see the staircase. When I was there people were milling around while a recorded message told the breathless story of the miraculous staircase. The story goes something like this. The chapel needed a staircase.  So some nuns prayed to the sky fairy of carpenters and low and behold a carpenter showed up with a bag of simple tools. Over the next month this remarkably skilled carpenter fashioned this beautiful wood staircase using only his simple tools. Apparently he constructed the staircase from the floor to the upper level without central supports. Even now the staircase lacks the standard central beam of spiral staircases. The masses were amazed! How could the staircase stand without a central support? Surely this is the work of the divine. This is what passes for a miracle among sky fairy believers. It’s the same type of magical thinking that invokes aliens to explain the pyramids. Just maybe the carpenter knew what he was doing and had the technique to pull it off. Occam’s razor people: it cuts deep.

The Santa Fe Trail

Posted on by
Reply

In the last ten years, we’ve moved five times.1 In a few days, I will increment that count. We are moving to Santa Fe New Mexico. Our previous perambulations were driven by work. We went where the jobs were and they were all over.

Because I have spent a lifetime moving for work I have no patience or sympathy for people who insist on staying put.

“I can’t leave here because … blah, blah, blah,” said every lazy whiner ever.

If you have to move to find a job get off your fat ass and move. Humans evolved on the move. Our distant ancestors trekked great distances foraging and hunting. We are happiest when moving. Putting down roots is for plants, not people. I’ll eventually settle down when I’m dead. Until then I am on the move.

Moving is old hat; what’s different this time is picking a place first. For over twenty-five years I’ve been a wandering software developer. This is beyond idiotic. The most portable stuff in the world is software. The internet makes it possible to create software anywhere and deploy it everywhere almost cost-free. There’s no need to pile programmers into pits to extract bits. Theoretically, all software developers could work remotely, but in case you haven’t noticed, we don’t live a theoretical world.

It’s easier for management to impose discipline and administer idiocies like SOX compliance if uppity developers are in the same room. Remote discipline is too easily mitigated with the mute button meaning management must come up with good ideas to herd their far-flung cats. Some enlightened outfits successfully manage productive remote developers but frankly most are struggling or openly hostile to the idea.

When it first occurred to me, way back in the 1980s, that commuting to an actual office was unnecessary I thought that within a decade or two most software development organizations would embrace remote work. The cost savings and enhanced access to global talent seemed like total no-brainers. Well, for many reasons, some good, (like getting smart people together), and some stupid, (see SOX), the brave new world of ubiquitous remote workers remains an infuriating work in progress.

I expect this muddle to eventually resolve but in the meanwhile, I am no longer willing to abide locales that do not suit me. So, for this move, we picked a spot, Santa Fe, that mostly meets our esoteric geographic preferences and then started looking for a job. Santa Fe is a small city so it took a little longer to find a good job but this thing called the internet also simplifies job hunting.

I’ve enjoyed my Saint Louis sojourn but like many Missourians of the 19th century, it’s time to head down the Santa Fe Trail.

Casa Joma

We are building a little house in the hills near Santa Fe. We will have unimpeded views of the mountains to the north. The lack of street lights, combined with Santa Fe’s 2000 meter elevation, should make for decent stargazing.

  1. Frequent moves are a good way to dodge jury duty.↩

On Shiny Pony and Witch Berning

Posted on by

This week Canada’s shiny new Prime Minister Justin Trudeau is hanging out with Obama. You can imagine my excitement: two photogenic lightweights sharing a heartwarming bromance on the public’s dime. I am at a loss to imagine a more hurl inducing scenario. I shared my feelings with a terse CBC comment.

As a dual US-Canadian citizen I must avert my eyes when confronted with such unctuous and expensive dog and pony shows. At least, we know who the dog is and who the shiny pony is! In a few short months, the toilet of history will flush on the Obama administration. For this we can thank term limits: one the best ideas in government ehh! This will cure the American Obama problem and ruin Trudeau’s play dates for years to come as I’m pretty sure neither Hillary or Trump are compatible with whatever is putting the wave in Justin’s hair.

I have safely ignored Justin Trudeau, (Shiny Pony), for decades and no harm will come from continuing this practice. Canada is, in Douglas Adam’s immortal words, “mostly harmless.” It’s one of the things I love about the country. It balances out the smug, preening, holier than thou, moral posturing that erupts from many Canadians when presented with conundrums like Trump. Batman’s nemesis the Joker said it best, “Why so serious?” Canadians take themselves more seriously than the Danes. If you’ve ever lived in Denmark you’ll understand.

Unfortunately, the United States is not mostly harmless. It matters a little bit who is elected President. It doesn’t matter nearly as much as many Americans think and we can praise the all squiggling Flying Spaghetti Monster for that. Living under a hubris-infused narcissist like Obama would be far worse if we didn’t have the luxury of ignoring him. And this brings me to Hillary, (Hildabeast), and Trump. It’s going to be difficult to ignore either one of them, and sadly None of the Above, my preferred candidate, is not on the ballot. This is a problem.

My solution, as always, revolves around who I hate the most. Animus is my animating principle. Trump is a bombastic tool, but, at least, I don’t want to hold his head under water until it stops making nagging noises. The prospect of four years of Hildabeast barking is profoundly depressing. The bitch must be stopped so I’m resorting to extreme measures. Next week I’ll cast a primary vote for that old 1960s era western hippie-commie, Bernie Sanders. Sanders has no chance of winning the presidency in November. If by some miracle, he ends up as the Democrat nominee he will be crushed by whatever crawls out of the Republican swamp.

I almost feel sorry for Bernie. Like all 1960s era western hippie-commies, he’s a weird mixture of earnest naiveté and historical ignorance. Western hippie-commies differ from their real eastern counterparts. They’ve never held power, never run anything of importance, and have a serious marketing problem!

Anyone, even remotely cognizant of 20th-century history, will find it hard to overlook the vast pile of corpses various communist and socialist regimes piled up. We’re not really sure how many tens of millions died at the hands of Stalin, Mao, Kim Il-Sung, Pol Pot and lesser thugs but it exceeds the toll exacted by the Nazi’s. This is beyond historical dispute. You can whine, attempt to change the subject, and hurl as many epithets as you want but it won’t change this nasty fact. People’s regimes are demonstrably bad for people.

Western hippie-commies like Sanders know this so they rebrand themselves as socialists. Socialists believe that if you only do communism right none of the brutal side effects will accrue. You can have somebody else’s cake and eat it too. This is naïve. Ask anyone in Venezuela about the wonders of 21st-century socialism. While you’re at it, explain how the socialism of Bernie Sanders differs from the socialism of Hugo Chavez. Spoiler alert: it doesn’t!

A primary vote for Sanders is mostly harmless. It’s like sticking a cocklebur on Pussydent Hildabeast’s hideous hide. Still it amuses me? I must take solace somewhere. Let’s Bern the Witch!

bernthewitch

SWAG a J/EXCEL/GIT Personal Cash Flow Forecasting Mob

Posted on by
Reply

While browsing in a favorite bookstore with my son, I spotted a display of horoscope themed Christmas tree ornaments. The ornaments were glass balls embossed with golden birth signs like Aquarius, Gemini, Cancer, et cetera, and a descriptive phrase that “summed up” the character of people born under a sign. Below my birth sign golden text declared, “Imaginative and Suspicious.”

I said to my son, “I hate it when astrological rubbish is right.”

I am imaginative and suspicious; it’s a curse. When it comes to money my “suspicious dial” is permanently set on eleven. I assume everyone is out to cheat and defraud me until there is overwhelming evidence to the contrary. Paranoia is generally crippling but when it comes to cold hard cash it’s a sound retention strategy.

Prompted by an eminent life move, I found myself in need of a cash flow forecasting tool. Normal people deal with forecasting problems by buying standard finance programs or cranking up spreadsheets; imaginative and suspicious people roll their own.

SWAG

SWAG, (Silly Wild Ass Guess), is a hybrid J/EXCEL/GIT mob1 that meets my eccentric needs. I wanted a tool that:

  1. Abstracted away accounting noise.
  2. Was general and flexible.
  3. Used highly portable, durable, and version control friendly inputs and outputs.
  4. Reflected what ordinary people, not tax accountants, actually do with money.
  5. Is open source and unencumbered by parasitic software patents.

Amazingly, my short list of no-brainer requirements eliminates most standard finance programs. Time to code!

SWAG Inputs

The bulk of SWAG is a JOD generated self-contained J script. You can peruse the script here. SWAG inputs and outputs are brain-dead simple TAB delimited text tables. Inputs consist of monthly, null-free, numeric time series tables, scenario tables, and name cross-reference tables. Outputs are simple, null-free, numeric time series tables. Input and output time series tables have identical formats.

A few examples will make this clear. The following is a typical SWAG input and output time series table.

 Date        E0      E1       E2      E3  E4     E5  E6  E7  E8  EC  EF  Etotal   I0       I1  I2  I3  I4  I5  IC  Itotal   R0         R1        R2  R3  Rtotal     D0  D1  D2  D3  D4  Dtotal  BB       NW         U0         U1  U2  U3
 2015-09-01  912.00  1650.00  100.00  0   50.00  0   0   0   0   0   0   2712.00  4800.00  0   0   0   0   0   0   4800.00  130000.00  25000.00  0   0   155000.00  0   0   0   0   0   0       2088.00  157088.00  155000.00  0   0   0 
 2015-10-01  912.00  1656.88  100.00  0   50.00  0   0   0   0   0   0   2718.88  4806.00  0   0   0   0   0   0   4806.00  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2087.13  159291.79  0          0   0   0 
 2015-11-01  912.00  1663.78  100.00  0   50.00  0   0   0   0   0   0   2725.78  4812.01  0   0   0   0   0   0   4812.01  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2086.23  161378.02  0          0   0   0 
 2015-12-01  912.00  1670.71  100.00  0   50.00  0   0   0   0   0   0   2732.71  4818.02  0   0   0   0   0   0   4818.02  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2085.31  163463.33  0          0   0   0 
 2016-01-01  912.00  1677.67  100.00  0   50.00  0   0   0   0   0   0   2739.67  4824.05  0   0   0   0   0   0   4824.05  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2084.37  165547.70  0          0   0   0 
 2016-02-01  912.00  1684.66  100.00  0   50.00  0   0   0   0   0   0   2746.66  4830.08  0   0   0   0   0   0   4830.08  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2083.41  167631.12  0          0   0   0 
 2016-03-01  912.00  1691.68  100.00  0   50.00  0   0   0   0   0   0   2753.68  4836.11  0   0   0   0   0   0   4836.11  130054.17  25062.50  0   0   155116.67  0   0   0   0   0   0       2082.43  169713.55  0          0   0   0

The first header line is a simple list of names. The first name “Date” heads a column of first of month dates in YYYY-MM-DD format. The SWAG clock has month resolution and dates are the only nonnumeric items. Names beginning with “E” like E0, E1, …, are aggregated expenses. Names beginning with “I” like I0, I1, I2 … are income totals. “R” names are reserves: basically savings, investments, equity and so forth. “D” names are various debts. BB is basic period balance, NW is period net worth and “U” names are utility series. Utility series facilitate calculations. Remaining names are self-explanatory totals. Be aware that this table has been formatted for this blog. Examples of raw input and output tables can be found here.

The next ingredient in the SWAG stew is what many call a scenario. A scenario is a collection of prospective assumptions and actions. In one scenario you buy a Mercedes and assume interest rates remain low. In another, you take the bus and rates explode. When forecasting I evaluate five basic scenarios, grim, pessimistic, expected, optimistic, and exuberant. Being a negative Debbie Downer type I rarely invest time in exuberant scenarios. I concentrate on grim and pessimistic scenarios because once you are mentally prepared for the worst anything better feels like a lottery win.

The following is a typical SWAG scenario table. Scenario tables, like time series tables, are simple TAB delimited text files.

 Name         Scenario On Group       Value  OnDate     OffDate    Method   MethodArguments                                                Description                                                                     
 reservetotal s0          assumptions 0      2015-09-01 2015-10-01 assume   RSavings=. 0.5 [ RInvest=. 3 [ REquity=. 3 [ ROther=. 1        annual nominal percent reserve growth or decline during period                  
 car          s0                      50     2015-09-01 2035-08-01 history                                                                 annualized car maintenance until first death                                    
 house        s0                      912    2015-09-01 2016-08-01 history  BackPeriods=.1                                                 current rent until move                                                         
 insurance    s0                      100    2015-09-01 2035-08-01 history                                                                 car insurance                                                                   
 living       s0                      1650   2015-09-01 2044-01-01 history  YearInflate=.5                                                 normal monthly living expenses                                                  
 salary       s0                      4800   2015-09-01 2016-08-01 history  BackPeriods=.4 [ YearInflate=.1.5                              maintain net monthly income until move                                          
 reservetotal s0                      25000  2015-09-01 2015-10-01 reserve  Initial=.1 [ RInvest=. 1                                       stock value at model start                                                      
 reservetotal s0                      130000 2015-09-01 2015-10-01 reserve  Initial=.1                                                     savings at model start                                                          
 salary       s0                      4200   2016-08-01 2023-07-01 history  BackPeriods=.4 [ YearInflate=.1.5                              reduced net income after move until retirement                                  
 house        s0          move        2000   2016-08-01 2016-09-01 history                                                                 moving expenses                                                                 
 house        s0                      100    2016-08-01 2044-01-01 history                                                                 incidental housing expenses after move                                          
 house        s0                      100    2016-08-01 2044-01-01 history                                                                 home owners association payments                                                
 house        s0                      150    2016-08-01 2044-01-01 history                                                                 property taxes                                                                  
 reservetotal s0          buy house   110000 2016-08-01 2016-09-01 reserve  Initial=.1 [ REquity=. 1                                       down payment added to house equity initial setting prevents double spend        
 loan         s0          buy house   150000 2016-08-01 2023-02-01 borrow   Interest=. 4.5 [ YearTerm=. 30 [ DHouse=.1  [ LoanEquity=.1    30 year mortgage rate on house until inheritance                                
 reservetotal s0          buy house   110000 2016-08-01 2016-09-01 spend                                                                   down payment on house from savings                                              
 annuities    s0                      250    2018-07-01 2035-08-01 history                                                                 monthly retirement and other annuity payments end date is unknown               
 annuities    s0                      50     2018-07-01 2035-08-01 history                                                                 any government pension payments                                                 
 reservetotal s0          assumptions 0      2020-01-01 2044-01-01 assume   RSavings=. -2.5 [ RInvest=. -5.0 [ REquity=. -5.0 [ ROther=. 2 market tanks government introduces negative interest                            
 loan         s0          buy car     10000  2020-07-01 2025-07-01 borrow   Interest=. 5 [ YearTerm=. 5 [ DCar=.1                          pay balance of car at 5% for 5 years                                            
 reservetotal s0          buy car     7000   2020-07-01 2020-08-01 spend                                                                   car down payment from savings                                                   
 reservetotal s0          inherit     180000 2023-01-01 2023-02-01 reserve  Initial=.1                                                     inheritance to savings                                                          
 reservetotal s0          buy house   150000 2023-03-01 2023-04-01 transfer Fee=. 1500 [ DHouse=.1                                         pay off remaining mortgage balance after inheritance fee is closing cost        
 salary       s0                      1400   2023-07-01 2035-08-01 history                                                                 estimated social security payments spread over expected life                    
 insurance    s0                      700    2023-07-01 2024-12-01 history                                                                 medical insurance in the gap between retirement and spouse medicare eligibility 
 annuities    s0                      100    2024-12-01 2044-01-01 history                                                                 any retirement pension payments to spouse                                       
 annuities    s0                      100    2035-08-01 2044-01-01 history                                                                 any us social security survivor benefit after first death

Again the first header row is a simple list of names. Most scenario names are self-explanatory but four OnDate, OffDate, Method, and MethodArguments merit some explanation. SWAG series methods assume, history, reserve, transfer, borrow, and spend are modeled on what people typically do with cash.

  1. assume sets expected interest rates and other global assumptions for a given time period. SWAG series methods operate over a well-defined time period. The period is defined by OnDate and OffDate.
  2. history looks at past periods and estimates a numeric value that is projected into the future. Currently, history computes simple means but the underlying code can use arbitrary time series verbs.
  3. reserve manages savings, investments, equity and other cash-like instruments.
  4. borrow borrows money and sets future loan payments. borrow supports simple amortization loans but is also capable of reading an arbitrary payment schedule that can be used for exotic2 loans.
  5. transfer moves money between reserves, debts, expenses and income series.
  6. spend does just what you expect.

SWAG series methods adjust all the series affected by the method. As you might expect SWAG arguments methods are detailed. MethodArguments uses a restricted J syntax to set SWAG arguments. Argument order does not matter but only supported names are allowed. Many examples of SWAG MethodArguments can be found in the EXCEL spreadsheet tp.xlsx. I use EXCEL as a scenario editor. By setting EXCEL filters, you can manage many scenarios.

The final SWAG input is a name cross-reference table. It is another TAB delimited text file that defines SWAG names. You can inspect a typical cross-reference table here.

Running SWAG

To run SWAG you:

  1. Prepare input files.
  2. Start J, any front-end jconsole, JQT or JHS will do, and load the Swag script.
  3. Execute RunTheNumbers.
  4. Open the EXCEL spreadsheet swag.xlsx, click on the data ribbon and then press the “Refresh All” button.

Let’s work through the steps.

Prepare Input Files

By far the most difficult step is the first. Here you review your financial status which means checking bank balances, stock values, loan balances and so on. Depending on your holdings this could take anywhere from minutes to hours. I call this updating actuals. Not only is updating actuals the most difficult and time-consuming step it is also the most valuable. Money that is not closely watched leaks away.

I store my actuals in a simple tabbed spreadsheet. Each tab maintains an image of a text file. I enter my data and then cut and paste the sheets into a text editor where I apply final tweaks and then save the sheets as TAB delimited text files.

Monthly income, expenses and debts are easy to update but some of my holdings do not offer monthly statements. The verb RawReservesFromLast in Swag.ijs fills in missing months with the last known values. When I’m finished preparing input files I’m left with four actual TAB delimited files, RawIncome.txt, RawExpenses.txt, RawReserves.txt, and RawDebts.txt. You can inspect example actual files here.

Start J and load the Swag script.

The SWAG script is relatively self-contained. It can be run in any J session that loads the standard J profile. Load Swag with the standard load utility.

 load 'c:/pd/fd/swag/swag.ijs'

Here SWAG is loaded in JHS.

jhsswag

Execute RunTheNumbers

RunTheNumbers sets the SWAG configuration, loads scenarios, copies actuals to each scenario, and then evaluates each scenario. Scenarios are numbered. I use positive numbers for “production” scenarios and negative numbers for test scenarios. It sounds more complicated that it is. This is all you have to do to execute RunTheNumbers

 RunTheNumbers 0 1 2 3 4

The code is simple and shows what’s going on.

RunTheNumbers=:3 : 0

NB.*RunTheNumbers v-- compute all scenarios on list (y).
NB.
NB. monad:  blclFiles =. RunTheNumbers ilScenarios
NB.
NB.   RunTheNumbers 0 1 2 3 4

NB. parameters sheet is the last config sheet
ModelConfiguration_Swag_=:MainConfiguration_Swag_
parms=. ".;{:LoadConfig 0
scfx=. ScenarioPrefix

ac=. toHOST fmttd ActualSheet 0
ac write TABSheetPath,'MainActuals',SheetExt

sf=. 0$a:
for_sn. y do.
  ac write TABSheetPath,scfx,(":sn),'Actuals',SheetExt
  sf=. sf , parms Swag sn [ LoadSheets sn
end.

sf 
)

RunTheNumbers writes a pair of TAB delimited forecast and statistics files for each scenario it evaluates.

Open swag.xlsx and press “Refresh All”

The spreadsheet swag.xlsx loads SWAG TAB delimited text files and plots results.3 I plot monthly cash flow, estimated net worth and debt/equity for each scenario. The following is a typical cash flow plot. It estimates mean monthly cash balance over the scenario time range.

meanbalance

The polynomial displayed on the graph is an estimated trend line. Things are looking bleak.

Here’s a typical net worth plot.

networth

In this happy scenario, we die broke and leave a giant bill for the government.4

So far SWAG has met my basic needs and forced me to pay more attention to the proverbial bottom line. As I use the system I will fix bugs, refine rough spots, and add strictly necessary features. Feel free to use or modify SWAG for your own purposes. If you find SWAG useful please leave a note on this blog or follow the SWAG repository on GitHub.

  1. What do you call dis-integrated collections of programs that you use to solve problems? Declaring such dog piles “systems” demeans the word “system” and gives the impression that everything has been planned. This is not how I roll. “Mob” is far more appropriate. It conveys a proper sense of disorder and danger.↩
  2. When borrowing money you should always plan on paying it all back. Insist on a complete iron clad repayment schedule. If such a schedule cannot be provided run like hell or prepare for the thick end of a baseball bat to be rammed up your financial ass.↩
  3. It may be necessary to adjust file paths on the EXCEL DATA ribbon to load SWAG TAB delimited text files.↩
  4. They can see me in Hell to collect.↩

What is Required: Print Captions on the back of Photographs

Posted on by

I am tired of waiting for “the market” to exploit painfully obvious opportunities so I will now provide guidance, in the form of milliblog entries, that tell our less imaginative entrepreneurs just what some of us would buy if we could.

Here’s a deep request. How about printing captions on the back of photographs?

I just went through the excruciating ordeal of ordering about one hundred prints from SmugMug. SmugMug prints meet my high standards but their online ordering software is clunky and clearly geared toward very small print runs. If you have hundreds of pictures, of varying and custom sizes, the software will punish you. The obvious steps of setting a paper type, selecting all your images, and then letting the software work out the paper size from image aspect ratio is not available. You must go image by image setting one easily derived parameter after another. I would print a lot more if it wasn’t such a frigging chore.

As annoying as SmugMug print-ordering is at least I get the prints I want with one major exception. If you check out my online photographs you’ll see I use captions like nanoblog entries. Many people have told me they really like my captions. One of my pet peeves is unlabeled photographs. I have wonderful hundred-year-old photographs of elegantly posed complete strangers because nobody left a clue on the back.

So, in addition to printing timestamps and file names on the back of photographs, include captions as well!

P.S. It takes about five years for the market to meet my obvious requirements; don’t hold your breath.