Back when I wrote about the CONS Parser project, I said that I wanted to add a function for exporting the solutions to the CONs so I could submit them for credit to the ACA (not a sponsor), but that I wasn’t going to do that right away. Actually, I’ve been waffling over exactly how I want to prioritize my time here.
Initially, I was thinking that I’d concentrate on adding GUIs to my existing solvers, and I’d only take credit for the CONs that I solved with the GUI-enabled solvers that were already finished. Then, I got the cryptarithm project pretty much fully working, and ended up solving all 16 CONs in the May-June issue of the Cm, as well as 14 or 15 of the CONs in the July-August issue, and it was getting close to the end of the August deadline for submitting the MJ2023 solutions for credit.
I figured I might as well tackle the solutions exporter feature right away, and as I thought about this, it seemed reasonable to go ahead and manually solve a few of the other ciphers in the MJ2023 issue just to have something to fully test the exporter on. Before I knew it, I had a full 76 CONs solved, making the need for the exporter that much greater. I ended up saving the manual solutions to a notepad .txt file, and then hand copy-pasted them into the database via the CONs Parser app. That last step took me about an hour, and manually solving the other 60 CONs took maybe 10 hours total. So, not too shabby. Gotta hope it goes even faster after I’ve added the GUIs to those other tools…
Anyway, I whipped up the exporter section in roughly 4 hours, and it worked well enough to justify using it to prep the contents of the email I sent to the newsletter editors for credit. Note that I’m not bothering with putting in a whole lot of error handling, because most of the problems we’d normally encounter would be due to operator error, or someone mis-editing one of the data files. Since I’m the primary user here, I’m not going to intentionally mis-edit anything.
Brief background:
The American Cryptogram Association‘s (ACA) Cryptogram (Cm) newsletter comes out every two months, and you get four months to solve the 100+ CONs (constructions; AKA – recreational cryptograms) each time. That is, the January-February issue usually hits the mailboxes a day or two before Jan. 1, and the deadline for emailing the solutions for it is April 30. The other issues are Mar.-Apr., May-June, July-Aug., Sept.-Oct. and Nov.-Dec.
The format for the email can vary based on the sender, but what I use is:
To: solutions_editor@aca.org (not the real address)
Subject: SOLs for issue, for NOM (my name)
issue AA PP XX EE CC SS Issue # YTD
name .nn nn nn nn nn nn .. # .. total
List of solved CONs (first five words each, and/or keyword if recovered)
Example:
MJ2023 AA PP XX EE CC SS Issue YTD
Chief. 12 10 -- 10 14 02 . 3 . 120
A-1 THERE ONCE WAS A MAN
A-2 FOUR SCORE AND A DAY .. Key: HYPERBOLE
etc.
Notes:
1) To save the running solutions totals, and the email filename template, I need to have a kind of .ini file that I can read and update when I do the export.
2) I need to add an Export option to the CONS Parser GUI menu bar.
3) If the .ini file doesn’t already exist in the Python files directory, I need the program to make a default.
4) I need to store the current year in the .ini, so that if the user (me) exports solutions for the following year, the exporter will automatically zero out the prior year’s per-issue solutions totals, and start over with a YTD of 0.
5) I’ve been seriously thinking about having a shared .ini for all of the GUI solver projects in preparation for when I turn everything into .exe files and still have the ability to customize the directory paths for where the digital CONs, the _processed.txt, and solved CON email files can go. I haven’t committed to this idea yet.
The format of the .ini file:
I’m calling the “.ini file” “yearly_stats.txt”, and the internal format is:
------- yearly_stats.txt -------
23
cryptogram_answers_###_email.txt
1 0
2 0
3 0
4 0
5 0
6 0
------ end of file --------
Line 1 is the last two digits of the current year.
Line 2 is the template for the solutions .txt filename I want to create.
Lines 4-9 store the SOL total for each issue for the current year.
I think this format is pretty self-explanatory. And the filename for the above MJ2023 solutions example would then be:
cryptogram_answers_03_MJ2023_email.txt
The changes to cons_parser_gui.py are:
A new import:
from cons_parser_credits import save_cons_for_credit
—
(File pull-down menu with Export option)
The addition to the file menu set-up:
file_menu.add_command(label='Export', command = lambda:....save_sols(root, CONS_FILE_PATH, entries))
—
The new function definition in cons_parser_gui.py:
def save_sols(root, init_dir, entries):
... save_cons_for_credit(CONS_FILE_PATH, root.con_records.filename,
....... root.con_records.records)
... popup_msg(root, 'File exported (probably).')
—
Then, we have the new cons_parser_credits.py module:
##########################
# cons_parser_credits.py #
##########################
The functions in this file are:
save_cons_for_credit(filepath, filename, records)
get_first_five(line)
get_type(con_no)
get_issue_no(fn)
get_to_date_total(yearly_stats, issue_no)
open_datafile(fn, default_monthly_fn, year)
save_datafile(fn, stats)
saveto_email_file(filepath, sols_email, email_fn, issue_no,... issue_total, to_date_total, sols_fn, nom, aa, pp, xx, ee, cc, ss)
"""
Imports
I need the ClassConRecords definition from cons_gui_utils, as well as the ConTitleFields enums.
I'll be doing some file I/O, so I want the Python Path module, too.
""""
from cons_gui_utils import ClassConRecords, ConTitleFields
from pathlib import Path
"""
Global variables
NOM is the user's pseudonym with the ACA.
SOLS_EMAIL is the email address for the Solutions department editor. If you are a member of the ACA, you can get this from the Cm.
YEARLY_STATS_FN is the filename for my ".ini file."
DEFAULT_STATS_FN is the template for the file I'm storing my SOLs email body to.
"""
NOM = 'The Chief'
SOLS_EMAIL = '###(at)cryptogram(dot)org'
YEARLY_STATS_FN = 'yearly_stats.txt'
DEFAULT_STATS_FN = 'cryptogram_answers_###_email.txt'
"""
save_cons_for_credit()
This is the main function called by the GUI.
filepath: Path to the digital CONs folder.
filename: Name of the current "_processed.txt" file.
records: The list object with all of the CONs, and solutions in the current "_processed.txt" file.
Take the current database records, and look for the ones marked "done". Separate the records into major CON types (Aristocrats, Patristocrats, Xenos, Cipher Exchange, Cryptarithms, Specials). Format the output and save to file.
Note that I could have done everything as a big 6xn list object array, which might have been somewhat cleaner, but this way I can keep track of what goes where based on individual list names.
"""
def save_cons_for_credit(filepath, filename, records):
... # Make the individual list objects.
... aa, pp, xx, ee, cc, ss = [], [], [], [], [], []
... # Grab the month and year from the filename.
... fn = filename[:6].upper()
... # Try to open the yearly stats file and return the contents as... # a list.
... yearly_stats = open_datafile(YEARLY_STATS_FN, DEFAULT_STATS_FN,....... fn[4:6])
... # Step through each record, looking for done == True
... for record in records:
....... if record[1][ConTitleFields.done.value]:
# Concatenate the CON number, the first five words of the solution
# and the key, if any. For cryptarithms, just use the CON number
# and the solved keyword(s).
........... con_no = record[1][ConTitleFields.con_no.value] + ' '
........... con_type = get_type(con_no.upper())
........... key = record[1][ConTitleFields.key.value]
........... if con_type == 4:
............... cc.append('%s %s' % (con_no[:4], key))
........... else:
............... if 'C-SP' in con_no.upper():
................... ss.append('%s %s' % (con_no[:7], key))
............... else:
# Add the concatenated data to the correct list.
................... short_str = get_first_five(record[3])
................... if len(key.strip()) > 0:
....................... short_str = (short_str + ' ' * 50)[:50] +........................... 'Key: ' + key
................... if con_type == 0: aa.append('%s %s' % (con_no[:4],...................... short_str))
................... if con_type == 1: pp.append('%s %s' % (con_no[:4],...................... short_str))
................... if con_type == 2: xx.append('%s %s' % (con_no[:4],...................... short_str))
................... if con_type == 3: ee.append('%s %s' % (con_no[:4],...................... short_str))
................... if con_type == 5: ss.append('%s %s' % (con_no[:7],...................... short_str))
... # Add up the total number of solved CONs for the issue.
... issue_total = len(aa) + len(pp) + len(cc) + len(xx) + len(ee) +....... len(ss)
... # Get the issue number (1-6)
... issue_no = get_issue_no(fn)
... # Add the current issue total to Yearly Stats.
... yearly_stats[2 + issue_no] = '%s %s' % (issue_no, issue_total)
... # Build up the YTD total from the yearly stats list.
... to_date_total = get_to_date_total(yearly_stats, issue_no)
... # Save everything to the email .txt file.
... saveto_email_file(filepath, SOLS_EMAIL, yearly_stats[1],....... issue_no, issue_total, to_date_total, fn, NOM, aa, pp, xx,....... ee, cc, ss)
... # Update the Yearly Stats .txt file.
... save_datafile(YEARLY_STATS_FN, yearly_stats)
"""
get_first_five()
Read the solution text for the current solved CON, and return the first 5 words from the line. Note that at least one CON had ". . ." in the solution, which needed to be included, but not counted as part of the word count.
"""
def get_first_five(line):
... # Split the line on the word spaces.
... line_list = line.split(' ')
... ptr = 0
... cnt = 0
... done = False
... while not done:
....... # The element is a "word". Count it.
....... if line_list[ptr] != '.':
........... cnt += 1
....... ptr += 1
....... # May have a short CON, or have 5 words already.
....... # Finish counting and return the results.
....... if ptr >= len(line_list) or cnt == 5:
........... done = True
... return ' '.join(line_list[:ptr])
"""
get_type()
Look at the prefix for the CON number to get the major type. If the CON number contains "-SP-", then add it to the Specials list. Anything that starts with letters other than those in the below if-statements will be treated as a special.
"""
def get_type(con_no):
... ret = 5
... if 'SP' in con_no: ret = 5
... elif con_no[:2] == 'A-': ret = 0
... elif con_no[:2] == 'P-': ret = 1
... elif con_no[:2] == 'X-': ret = 2
... elif con_no[:2] == 'E-': ret = 3
... elif con_no[:2] == 'C-': ret = 4
... else: ret = 5
... return ret
"""
get_issue_no()
Look at the first two letters of the filename and turn them into an issue number, 1-6.
"""
def get_issue_no(fn):
... ret = 1
... month = fn[:2]
... if month == 'MA': ret = 2
... elif month == 'MJ': ret = 3
... elif month == 'JA': ret = 4
... elif month == 'SO': ret = 5
... elif month == 'ND': ret = 6
... return ret
"""
get_to_date_total()
Take the Yearly Stats list, and count the issue totals from issue 1 up to the current issue number. That is, if the issue is JF2023, just sum the first issue total. If it's ND2023, sum everything from issues 1 to 6.
Note that the data in yearly_stats will be in elements 3-9, and will be strings in the form of "# total".
Split the elem on the space, and take the integer value of the contents in elem[1].
"""
def get_to_date_total(yearly_stats, issue_no):
... total = 0
... for i in range(issue_no):
....... elem = yearly_stats[i + 3].split(' ')
....... total += int(elem[1])
... return total
"""
open_datafile()
Try to read the Yearly Stats file named in "fn." If it doesn't exist, create and return a default template. Otherwise, take the last two digits of the current year given in "year," and compare it to line 1 of the file. If they don't match, assume that we're starting a new year and that all of the old issue totals need to be zeroed. Otherwise, split the string on '\n' and return the list object.
"""
def open_datafile(fn, default_monthly_fn, year):
... # Get the path object to the file in "fn".
... path = Path(fn)
... if path.is_file():
....... # It exists. Read it.
....... with open(fn, 'r', encoding='utf8') as file:
........... fstr = file.read()
....... file.close()
....... # Split string by lines.
....... file_list = fstr.split('\n')
....... if file_list[0] != year:
........... # The year in the first line doesn't match.
........... file_list[0] = year
........... # Zero out the old issue totals.
........... for i in range(6):
............... file_list[3 + i] = '%s 0' % (i + 1)
....... return file_list
... else:
....... # No file. Create the default version.
....... ret = '%s\n%s\n\n1 0\n2 0\n3 0\n4 0\n5 0\n6 0\n' % ........... (year, default_monthly_fn)
....... return ret.split('\n')
"""
save_datafile()
Save the Yearly Stats file. This is just a matter of opening the ".ini. file" for writing, and writing the stats list object as-is.
"""
def save_datafile(fn, stats):
... with open(fn, 'w', encoding='utf8') as file:
....... file.write('\n'.join(stats))
....... file.close()
"""
saveto_email_file()
Save the assembled CON data to be included in the body of an email to the solutions department editor. Use the format described above in the explanation section of this entry.
filepath: Path to the "_processed.txt" folder.
sols_email: Solutions editor email address.
email_fn: Filename to create for the email body contents.
issue_no: 1-6, based on the issue filename (JF, MA, MJ, JA, SO, ND).
issue_total: Total number of CONs solved for this issue.
to_date_total: Total number of CONs solved YTD.
sols_fn: "_processed.txt" filename, month and year (i.e. - MJ2023).
nom: The user's pseudonym.
aa, pp, xx, ee, cc, ss: Lists of CON solutions and/or keys.
"""
def saveto_email_file(filepath, sols_email, email_fn, issue_no,... issue_total, to_date_total,
... sols_fn, nom, aa, pp, xx, ee, cc, ss):
... # Build up the save file path and filename from the email_fn
... # template. Replace "###" with the issue month and year... # (i.e. - 03_MJ2023).
... savepath = '%s/%s' % (filepath, email_fn.replace('###', '0' +....... str(issue_no) + '_' + sols_fn))
... # Open the file for writing and save the CONs solution data
... # in the format described above.
... with open(savepath, 'w', encoding='utf8') as file:
....... # Write the email header data.
....... file.write('%s\nSOLs for %s, for %s\n\n' %........... (sols_email, sols_fn, nom))
....... file.write('%s AA PP CC XX EE SS Issue YTD\n' % (sols_fn))
....... file.write('%s %2s %2s %2s %2s %2s %2s %s %3s\n\n' %........... (nom, len(aa), len(pp), len(cc), len(xx),........... len(ee), len(ss), issue_no, to_date_total))
....... # Write the first five words of the solutions
....... # and/or the keywords for each major cipher type.
....... # Separate cipher types by a blank line.
....... for elem in aa: file.write('%s\n' % elem)
....... file.write('\n')
....... for elem in pp: file.write('%s\n' % elem)
....... file.write('\n')
....... for elem in xx: file.write('%s\n' % elem)
....... file.write('\n')
....... for elem in ee: file.write('%s\n' % elem)
....... file.write('\n')
....... for elem in cc: file.write('%s\n' % elem)
....... file.write('\n')
....... for elem in ss: file.write('%s\n' % elem)
....... file.write('\n')
....... # Close the file and we're finished.
....... file.close()
======
Next up, theory of transposition.
The Black Chamber 