Previous version (wiki parsing) was developed in Lua with aim to auto generate
PMWiki page markdown syntax. Wiki Parsing served its purpose well for a couple
of years to generate page documentation from FreedroidRPG(FDRPG) source data.

However, with the move away from PMWiki, usage of static webpage generation, as
well as changes to FDRPG data files, wiki parsing is showing it's flaws. This
issue is an idea proposal to replace the current mechanism and to define that
process.

a) Remove Wiki Parsing from current FDRPG source tree
b) Insert Player Guide Page Generation (PGPG) into web site source tree
- player guides are dependent on FDRPG tree but not required for the game
- generating these pages is a function of the web site
- output of parsing is used by the web site

c) Perform a clean-up of the code to reflect changes
- current parsing has some broken and misdirected links due to both src tree and
site changes
- output written to other folder in FDRPG source tree and must be manually moved
to web folders
- parsing output was to be markup/data - now used to generate data/template -
can be improved

The above steps may include a re-write with a change in language. Lua is okay,
but its usage in this application is difficult and unwieldy. Parsing/Generation
involves the cross-referencing of several aspects of game data. Consideration is
being given to a more capable scripting language such as Python (at least 3 for
future proofing).

Suggestions and comments welcomed.

Above text seems good. Nothing to 
disagree on my part.

A few concerns about docpad capacity to 
handle multiple-file changes, second act 
introdution, and human-interation. Don't 
get me wrong, I would be very happy if 
the script was ran monthly by cron and 
took care of everything (including 
commit), and I wouldn't mind if it had to 
be run and then comitted by someone. I 
guess that wasn't stated and that we need 
fluzz input about that.

Worries comes as the project goes 
forward, so all worries above are mostly 
anticipations of things which may raise 
along the path.

This was written on a cellphone so please 
excuse any typo from my part.

Best regards from Brazil,
Jesusalva

Just to clarify a couple of points...
Previous usage was done on a best-effort basis. There were discussions about how
to trigger (with makefile-like abilities) rebuilding needed pages when certain
files were touched in the source tree. This was never really resolved.

A cron script - or always build these pages seemed rather overkill as the data
files that form the basis of the player pages do not change much. Monthly/As
Needed/Manually - not sure. A workflow to update/change player guides would
require some thought (use case, update mechanism, et al).

As for delineating which Act/Scene/avocado where something belongs is not too
much of an issue (check path; if folder=actX; set var=X).

From Lisured - suggestion to fix misdirected links in markup text
../npc or root-relative URL (ie /player-section/guides/npc-guide/)

As you mentioned, there are 3 points:

1) When to generate the web pages ?
In my opinion, the 'player guide' is used by 'standard players', who are using
the current game release.
So the 'player guide' must be in sync with the current game release.
Consequently, it has to be generated during the 'release process' (either
automatically by a slave-worker or manually, it does not really care at this
point of the discussion).

2) Where to put the generator ?
Well, as it was said, the generator's input is in freedroid-src, but its output
has to eventually go in freedroid-website... There is a strong interdependence :
the generator has to be adapted after a change in the data source
(freedroid-src), but it also has to be adapted in case of a change in the
website structure (freedroid-website). So, one repo is not better than the other.
In such a case, the usual solution is to used an 'exchange file' in an 'exchange
format' which is 'agnostic' of the data structure "in" the game and the data
structure "in" the website.
Then, the generator should perhaps be split in two parts: one part in
freedroid-src, which writes out the exchange file, and one in the
freedroid-website, which reads it.

3) Which language ?
Well, I would be in favor of using a 'standard' interpreted language,
specifically if (a part of) the generator is in freedroid-website (there is no
compilation process during the website generation).
And, in my opinion, given the computations needed, python is indeed a better
choice. However, the 'clean' process would be to write a docpad plugin...

[just some thoughts...]

I like the idea of a docpad plugin. Action would be tied to the site, so long as
we continue using docpad this would be ideal. Python would be a strong
consideration for "future proofing".

So to address fluzz's Q#3 above - we'll start with a docpad plugin and proceed
from there.

I think the notion of an exchange file format would be most useful here. Code to
generate, for example, xml of all game data and a plugin that processes that
data to produce page output put sounds ideal. I do not recall any examples where
existing code needs special game-based rule processing other than dependencies
(e.g. to produce droid pages we also need item data to link to droid weaponry).
Exchange file is a workable solution.

Until we come up with a mechanism for when to execute update process, manual
intervention can be employed to generate the Player Guides from src code.

Off to dive into coffee-script...

If you are to write a docpad plugin, and so use javascript (or a derivate), JSON
is probably (or even: definitely ;-)) a better choice than XML.

xml was an example. I agree JSON would be more suitable.

On the topic of splitting up workload...
It occurred to me that the original wiki parsing was duplicating the effort of
the game. Both would parse the data files to produce data structures. Rather
than maintain separate code, I would like to add a flag to FDRPG command line
arguments (http://www.freedroid.org/doxygen/tools/cmdlineref.html) to do exactly
this - read data files and output data in JSON format.

PGPG could then call the game with this flag, read-in the JSON, re-format and
output Player Guide pages.

After review...

Nice thought about usage of some kind of exchange file format between source
data files and web generation. However, I think this adds a layer of complexity
to the overall scope. I am unable to justify the needed effort to design and
maintain this idea. I would like to proceed with best effort to proceed with
data structures defined in struct.h but in scripting language format.

I had an uncomfortable feeling with lua in terms of its
capability/functionality. This maybe attributable to the original design, but
the whole parsing mechanism in its current state feels "fragile". To improve
long-term maintenance I will be doing a rewrite in python. I will future proof
by using python version 3+.

Moving of the parsing code out of the FDRPG source tree I will leave for future
consideration.

Main Points:
- restore functionality
- make code resilient and maintainable

That's ok.
One step at a time, was used to say our past dear project leader.

Progress...
I have almost all the python code to parse game sources in [FDRoot]/data/base/
and [FDRoot]/data/storyline/act1. Only file remaining is the droid_specs.lua (a
little involved).

The method being used seems much more refined and code has improved readability.
In a way, I am glad I didn't pursue lua-based code any further.

After finishing code to parse droid_specs.lua, I'll post an RB with code. Code
will be python (hopefully a module) that will only have source parsing. Writing
the code for page output write will be the next step.

More to follow...