Assuming you have already installed TiddlyWeb and TiddlyWebWiki (http://tiddlyweb.peermore.com/wiki/) this is how to install the very latest development version of TiddlyDocs on your local machine :
curl http://svn.tiddlywiki.org/Trunk/contributors/SimonMcManus/tiddlyweb/tiddlydocs/install.sh >install.sh sudo sh install.sh cd tiddlydocs twanager server 127.0.0.1 8080 In your browsers then goto :http://127.0.0.1:8080/recipes/tiddlydocs/tiddlers.wiki
My name is Simon McManus. I work as a web developer. After recently attending a UKGovBarCamp I noticed that it was difficult to reuse parliament’s publications. I made a comment on a parliamentary blog post
which resulted in Richard contacting me via e-mail. The fact that I was able to comment in the first place has made it possible for me to speak to you now. Thank you for this opportunity.
The essential dissatisfaction I have with the parliament website is that the information is not being published for re-use. In this paper I will explain what I mean by this, why I believe it and offer some alternative solutions. I would be more than happy to come and discuss this with you further and would appreciate any feedback that you might have.
Executive Summary
Websites like Wikipedia demonstrate how conversations can take place around information. For each article there is a discussions tab which allows readers and authors to discuss the articles. If you would like the same thing to occur around your meeting transcripts and legislation you need to change the format to make the data referenceable, commentable and easily queried by a programming language.
I believe there are five steps to opening up Parliamentary data:
1 . Data Copyright
2 . Making raw data available
3 . Marking up data with semantic information
4 . Making data linkable
5 . exposing data through APIs
each of these parts will now be discussed.
::: 1 . Data Copyright
Publishing data online holds little value if the data has not been licensed for reuse. I suggest all parliamentary publications be made available under a creative commons copyright, so that anyone can republish commercially or otherwise.
::: 2 . Making Raw data available.
It is important that the raw data dumps which make any application possible are also available to the public. The data should be available be with no style information, no scripting nothing but pure, unadulterated data. While there is value in Parliament building websites/applications it is far more important that developers have equal access to the original data so they can build other applications without the unaffected by the preconceptions of parliament.uk developers.
Currently most parliamentary publications are in PDF form. This causes a number of problems :
1 .. Individual pages and sections are not indexed by search engines.
2 .. It is difficult to programmatically extract data from a PDF.
3 .. It is not possible to reference particular sections of a document.
::: 3 . Marking up data with semantic information
Theyworkforyou.com have put together a basic template of how parliament can improve the semantics of parliamentary publications. More details of their suggestions can be found at the following address :
http://www.theyworkforyou.com/freeourbills/techy
I fully endorse these suggestions. If followed, they would make it a great deal easier for developers like me to build new and richer interfaces because it makes the data more meaningful.
::: 4 . Making data linkable
When writing the paper it was particularly difficult to find the references from transcripts of your Committee’s meetings. It was sent to me in the following form :
“The transcript of the meetings the Committee has had as part of its inquiry are available here:
http://www.publications.parliament.uk/pa/ld/lduncorr.htm#info
(See in particular questions 78, 85 and 86 of the 1 April meeting).”
Finding the information required the following steps to be taken:
1 .. clicking the above link
2 .. Searching for the meeting on the 1 April
3 .. clicking another link which downloaded a big PDF file
4 .. Waiting for the entire document to download
5 .. Searching the pdf for question 78
6 .. Searching the pdf for question 85
7 .. Searching the pdf for question 86
I would like to see an implementation where clicking the following three URLs would take you straight to view each question, allow you to read its answer and comment against either.
1 .. http://www.publications.parliament.uk/InformationCommittee/PeopleAndParliament/Meetings/1April09/Question78
2 .. http://www.publications.parliament.uk/InformationCommittee/PeopleAndParliament/Meetings/1April09/Question85
3 .. http://www.publications.parliament.uk/InformationCommittee/PeopleAndParliament/Meetings/1April09/Question86
If the information is published in HTML files which are being indexed by Google the bills will be findable in google and extend your outreach to every single user of google.
The simplest way to expose data on the web is to break it down into small individually addressable sections each of which has a unique URL. These URLs can then be sent round in emails, added to a user’s favorites or programmatically interrogated.
::: 5 . Exposing data through APIs
An Application Programming Interface (API) provides developers an interface for interacting with a data set easily. By making it possible to programatically search legislation and comment against a particular section from a remote site, it becomes much easier for people to build new interfaces for the available data. A good API would make it really easy to build new ways of browsing, searching and commenting on legislation.
Data should be exposed so that it can be presented in ways never expected by those collating the data. It is through this approach that you help people to view and, most importantly, interact with both Houses regarding proposed legislation.
A good API will make data available in a number of different formats. HTML, XML and JSON are a good starting point. From the earliest possible opportunity any code being used to expose data should be open sourced so that developers can extend the existing code base without needing to start from scratch. Not only does this allow people to build things more quickly, it allows developers to extend functionality and form a community of developers working together to improve the nations data infrastructure.
If Parliament wants to engage with people it will be a great deal easier on sites they already visit rather than the parliament.uk site. You cannot expect to engage the majority of the electorate at parliament.uk. It needs to be made particularly easy to integrate the goings on of both Houses into any website so that useful (relevant) data can be pulled in about a given subject.
A site about digital rights and copyright should be able to make a call to the API which looks for any recent mentions of “Digital Rights” and “Copyright” and can then embed the results in its own site. I also suspect that providing functionality to comment against the results would massively increase the potential of both Houses to engage with the electorate.
Below I have put together a general criteria for exposing data on the web :
1 .. The data should be indexable and discoverable in google/other search engines.
2 .. The data should be exposed using open data formats (HTML, oof)
3 .. The data should be licensed for re-use, even by commercial organisations.
4 .. It should be possible to browse the data in a web browser.
5 .. Make the original data set available.
6 .. Maintain a consistent interface for developers to build against.
7 .. Any code used to abstract away from the data should be open sourced.
8 .. Any semantics that can be added to data are beneficial.
9 .. There should not be separate systems for MPs and the public. Whatever system MPs use to look up bills, track their progress through parliament, find out amendments etc should be available to the public.
The following criteria are not essential but I suspect could have a major effect on parliaments ability to interact with the people online :
1 .. the information should have a plain English summary. Often bills are full of jargon that makes them incomprehensible to many. A good example is the creative commons licenses. These have a plain English version and a legalspeak version.
2 .. Electronic book support. All bill, minutes etc should be available in a non-proprietary electronic book format (eg epub) for download.
Please note all that all the above should be possible for very little cost. All the software required is available for free with open source software licenses. The primary cost should be for one or two developers who work with the community to expose data based on user/developer feedback.
Recently I’ve been building a proof of concept, the brief was simple…
“Come up with a way for my team to collaborate with people outside the organisation on large Word document. At the moment, we tend to email them between us, which can degenerate into a nightmare.”
Our approach was to think about breaking these large documents down into small sections which could be assigned to individuals for authoring and/or review. These sections could then be reassembled into a single document for printing.
To build this proof of concept we used TiddlyWiki and jQuery. The latest release of TiddlyWiki (2.5) includes jQuery (1.2.6) so this provided a fine opportunity to explore the new potential.
So here is the demo,
The online demo does not demonstrate the iGoogle integration of TiddlyDocs. The iGoogle integration showed how we could provide a personalised view of the sections assigned to each user. While we used iGoogle for the original demo this portlet just contains HTML so could be embedded into any portal framework which supports HTML.
PSD wrote some XSL-FO to translate the generated HTML into a printable PDF document. Paul’s code can be found here.
Drag and Drop your Table of Content
Allows users to change the order and hierarchy of the document table of content
Easily assign sections to Users
Allows each section to be easily assigned to a pre-populated list of users.
Full WYSIWYG text editing with FCKEditor
Allows users to edit the document content in a familiar word style environment. Also allows content including images to be pasted in the textarea from the clipboard.
iGoogle Integration of Personalised Task Lists
The iGoogle integration allows each users to have their own view of all the tasks assigned to them. This can be used in various portal frameworks.
Easy view of the documents status
The bar to the right of the table of content shows the status of each section. When the bar is completely green the document is complete. This example demonstrates that only one of the sections has been completed.
Each Section Allows for Comments
Each section allows multi-threaded comments which will not be printed in the final document. This allows for conversations to take place around each section of content.
The Components :
We were able to build this proof of concept so quickly by reusing open source code that is available in the TiddlyWiki and jQuery communities. Below I talk about the components that make up TiddlyDocs.
SplitView Plugin
This is one of the plugins I wrote specifically for the demo. The macro accepts one parameter which is the name of another tiddler to store the document specification. The document specification is stored in list format, eg:
* Heading 1
* Heading 2
** Heading 2.1
** Heading 2.2
* Heading 3
Every time the document is re-arranged the specification tiddler is saved.
PrintView Plugin
This is the other plugin which I wrote specifically for the demo. This plugin can read the document specification, collect the data from each of the required tiddlers, and then generate a static html file on the server which can then be converted to PDF for printing.
Single Page Mode Plugin
Provided by Eric of TiddlyTools, the singlePageModePlugin ensures only one tiddler is viewed at a time.
http://tiddlytools.com/#SinglePageModePlugin
Comments Plugin
Allows users to post comments against individual sections. These comments are presented as a threaded discussion, allowing rich conversations to take place around the content. Thanks to Michael Mahemoff for this plugin who also helped with the demo.
http://tiddlyguv.com/CommentsPlugin.html
Value Switcher Plugin TeamTasks
Taken from Phil Hawksworth’s TeamTasks, this allows the fields provided by TiddlyWiki to be used to store information about ownership and current status.
from: http://getteamtasks.com
FCKEditor
FCKEditor is a project to make a nice WYSIWYG editor from any HTML textarea. I have to say a I have been very impressed with FCKEditor.
TiddlyWiki FCKEditor Plugin
A TiddlyWiki plugin that makes FCKEditor work nicely with TiddlyWiki.
http://visualtw.ouvaton.org/VisualTW.html#FCKeditorPlugin
NestedSortable -(a jQuery plugin)
This jQuery plugin provides the drag and drop interface for the demo.
http://code.google.com/p/nestedsortables/
This demo is hosted by ccTiddly but its been written to work on TiddlyWeb and any other server-side implementation of TiddlyWiki. Over the next few months I will be working on TiddlyDocs to prepare it for easy consumption.
Updated : Michael Mahemoff has produced a screencast talking through the features of this demo : http://vimeo.com/3109248
Sorry I haven’t blogged for a while…I’ve been keeping myself busy with several changes to ccTiddly!
I’m pleased to announce the imminent release of ccTiddly v1.7! This version of ccTiddly was originally going to contain the new server side plugin functionality. It was decided that the next release would contain so much new functionality that it would be inappropriate and in fact confusing to release it as v1.6.5, and so v1.7 does not include server side plugins. Plugins are still planned for the next release: ccTiddly v1.8.
From v1.7 of ccTiddly onwards, users will be able to take a copy of a ccTiddly workspace (collaborative wiki) with them on a train, plane or anywhere without an internet connection. While a user is offline the wiki will save changes locally like a conventional TiddlyWiki file.
When the user has finished making changes offline, and when reconnected to the internet, they can click the sync button on their local version of ccTiddly and their changes will be sent back to the server.
Based on the changes, TiddlyWiki will show the following options before syncing:
This is very new functionality. It’s working in Firefox nicely. Unfortunately, I now need to get it working in IE6. Given that IE6 is likely to be challenging, I don’t want to make any promises about release dates. However I suspect it will be out in the next few weeks! If you can’t wait to have a play the code is in the TiddlyWiki subversion repository.
By using a TiddlyWiki adaptor, ccTiddly can now provide support for enhancements including lazy loading (see below) and integration with projects such as TiddlySnip (a firefox extension for moving web content to your wiki).
As part of the ccTiddly project, I have been rewriting many of the ccTiddly macros so that they work with TiddlyWiki slightly better than they did originally.
ccTiddly 1.7 will introduce the following new macros :
<<ccLogin>>
Ok, so this is not a new macro but it’s been completely redesigned and making full use of the TiddlyWiki wizard framework. It has also been integrated with the register macro. If the register macro is installed there will be a register button on the login screen.
I hope to extend this mechanism for things like OpenID login.
<<ccEditWorkspace>>
This macro will allow users to update the permissions on an existing workspace. It currently only allows users to set permissions for anonymous users but will be extended to support setting permissions for admins, logged in users and possibly groups.
<<ccManage>>
The ccManage macro will allow workspace administrators to grant other users administration access to a given workspace.
TiddlyWiki recently introduced the concept of single tiddler themes. These allow developers to pack many different tiddlers into a single tiddler which makes up a theme e.g. stylesheet, pageTemplate and viewTemplate. ccTiddly now makes full use of the theme mechanism. We will distribute ccTiddly 1.7 with three themes.
We have now reached an important milestone in ccTiddly where the Javascript and PHP code are completely separate. This greatly simplifies ccTiddly. All modification to theTiddlyWiki file are loaded from the /tiddlers/ folder as either .js files or .tiddler files.
There is a file: /includes/ccAssignments.php which creates the ccAssignments tiddler. This is the only point where the PHP and Javascript cross over.
Any requests made from the client to the server are handled by a file in the /handle/ folder.
With “Lazy loading” a subset of tiddlers are loaded in the first instance, and then others are only loaded upon request.
In ccTiddly, it’s possible to pass the tags variable in the URL. If the tags variable is specified in the URL, the ccTiddly workspace will only load tiddlers with the specified tag. For example;
tags=task
…would only return the tiddlers tagged ‘task’ in the requested workspace. If users click on links to tiddlers which are not tagged task, they would be loaded at that point.
This technique can be used to dramatically improve loading times of TiddlyWiki files. As soon as ccTiddly v 1.7 is available I will announce it on the ccTiddly groups.
I used to love politics. Over the last four or five years I’ve developed a dislike for it. I was starting to think I had maybe grown out of it or maybe I’m no-longer so idealistic and naive.
Some interesting things have been happening recently, I bumped into @absond on the train into work a good few months back. He told me about BarCampUKGovWeb which sounded quite interesting.
Ive since started following a number of blogs from people that work in and around government. I’ve even started to follow the blogs of some MP’s. I have so far followed all MP’s I am aware of regardless of party.
I don’t pay particular attention to every post but rather scan read whatever they choose to blog. As more MP’s start to blog I will need to become more selective. At the moment its interesting just to see what our representatives feel comfortable blogging about.
The other day this post from Lynne Featherstone caught my attention. She talks about human rights in China in relation to the Olympics. An issue at the front of my mind after going to see the Olympic torch pass St Paul’s the previous week. We didn’t actually see the torch, it was bundled onto a bus with the Chinese thugs just before it got to us. Regardless I found Lynne’s post quite agreeable.
Normally in this situation I would drop a comment on the blog to show my agreement. Unfortunately Lynne’s blog is not quite a real blog. You can’t add comments. Here is what is really cool though, Lynne’s on Twitter. So I tweeted her.
The Downing Twitter feed is rather interesting. @DowningStreet. It provides updates, responds to questions and actually provides an interesting insight into life at Downing Street.
The @Conservative twitter feed even impressed me yesterday. I was getting ready for work when they tweeted that David Cameron was going to be interviewed on BBC 1 just after 8am… and they wanted feedback. So I watched. And then I replied :
“@Conservatives - not bad – Your still not getting my vote. Just to confirm. Are you left or right these days?”
It was slightly sarcastic and I wasn’t particularly expecting a reply. When I arrived at work an hour later there was a reply from the Conservative Party.
“@simonmcmanus Thanks for your feedback, and we’re on the centre-right!”
For the first the first time in my life I had engaged in a two way conversation with a political party!
The 140 character constraint removes some of the formality associated with blogging and email. I sure as hell wouldn’t have written that in an email. Even an email of the same length. Twitter, it would seem is making politicians more accessible. Or at least making politicians appear more accessible.
I’m really interested to see how this all develops. I’m loving the idea of my MP being on Twitter. Lets hope Eleanor Lang (my MP) is listening. I would love to see politicians adopting these tools because they provide a highly effective communications platform.
I should thank http://puffbox.com/ for keeping me informed about the latest politicians using Twitter. Interesting to see that @brianpaddick , @backboris are both using Twitter.
Politics and politicians can often seem invisible and unaccountable. Twitter could start to change that. Given that MP’s claim to represent us in parliament I don’t think its wrong to wonder…..What are they doing?
The software produces great reports but the experience to the end user is rarely properly considered. The tendency is to meet the objectives of the business owner rather than the needs of end users.
So this blog post is about saving money. Well, time is money…right? Take an expenses system. If your employees spend their time trying to work out how to use the system, that’s time not being spent making money.
Imagine saving every employee 5-10 minute each month. Across 100,000 people, that’s 100,000 – 200,000 man hours a year!
Enterprise software pays a reasonable amount of attention to accessibility but not enough to usability. Now I’m no user experience expert but I have been exploring some experience design techniques with Phil recently (for the new ccTiddly macros). I think the correct use of such techniques within the enterprise would make a massive difference. In fact any technique that stops developers thinking so much like developers and more like the poor sod who’s going to be relying on the software the better.
Imagine being able to apply the time savings across each of JP’s four pillars : Search, Syndication, Conversation and fulfillment.
For those of you that have not already seen http://www.theyworkforyou.com/ please go and check it out now. Its been around for a good few years now but I was particularly pleased to see their recent action list (Free our Bills) has gained support from David Cameron. Im sure this will make lots of the “Give us our data back” folk happy.
In relation to Emma Mulqueenys point about Links, libel and law. My linking to Mr Cameron does not in anyway suggest support for his views except in the Free our Bills case. I am simply linking to an article which I have read/watched and found sufficiently interesting to share with others.
Over the last couple of days I have been working on the ccUpload macro for ccTiddly. This is a macro that will allow people to upload files to the ccTiddly server so that it can linked to and included in their TiddlyWikis.
The work was in part motivated by Jon’s requirement to post HTML files and strings to a end point where the file is then uploaded. This is a part of his template work and some SEO work which will naturally fall out from it – that, however, is a completely different blog post.
We are getting to the stage where we have something functional to share and wanted to provide the opportunity for community feedback.
We’re introducing three concepts for ccTiddly users; instances, workspaces and user areas.
Instance: An area containing multiple workspaces.
Workspace: An online area where an individual TiddlyWiki can be stored and maintained. The area can also contain files which have been uploaded by a workspace administrator.
User area: An online area exclusively dedicated to the uploading of files by a single user. Each user has their own user area by default.
The macro enabling file upload does not require the page to refresh in order to upload additional files; instead it posts the form to a hidden iFrame for processing. At present, we’re giving users the choice to upload their file to either their workspace or their user area. I am probably going to remove the ability to upload a file to a user area for sake of simplicity (although the code will still exist and can be used if needed).
I am yet to decide what convention I will use when assigning a user area directory to OpenID users. The following structure is not an option:
wiki.osomsosft.com/uploads/users/http://simonmcmanus.com
I would be interested to hear any suggestions/examples.
I have currently not implemented any security with ccTiddly. I am currently thinking that we will only allow people to upload files if they are the owner of the workspace; at the back end users will be able to upload files to their user space provided they are logged in.
Here are the screenshots showing the work so far….. feedback is welcomed.
You can either leave your comments against this blog post or leave them on the google groups page :
http://groups.google.com/group/ccTiddly
We are also going to need an area where people can browse and manage their uploaded files. That will probably take the form of a separate macro (ccBrowse??) but I would be interested to hear from anyone who sees value in displaying the previously uploaded files underneath the upload functionality as the below sketch shows :
Does this sound useful to anyone?
When I have formalised the upload API, I will document it on the ccTiddly page on TiddlyWiki.org. I hope to have this code released in the next week or two.
For me the bigger issue here is how we can encourage management to let people blog about their work. Particularly people working for Government. I suspect governance/regulation can only provide minimal assistance on this matter, what we need is common sense.
Personally I have never read our companies’ blogging policy, I think we have one, but there are some pretty common sense rules which have managed to keep me in work so far :
- Be authentic and honest.
- Don’t bitch – If you must then make sure you back it up with evidence.
- Justify what you say.
- Make your Mum proud, my mum doesn’t understand most of the stuff I blog about but I like to think she would be proud if she did understand.
- Learn from your mistakes.
- keep secrets secret.
- Talk to your boss about what you’re blogging and why.
- Abstract away from events/people/companies/parties and talk about anonymous examples.
- Don’t be afraid of being wrong (this one does depend a bit more on your employer)
- Make it clear that the views expressed are yours and not that of your past, present or any future employer.
I am quite fortunate in that my boss and his boss are bloggers with a good understanding of the blogosphere.
There is one key thing here, the people at the top need to realise that the “one voice culture” cannot be maintained. There will always be a variety of opinions in any organisation; it’s often the variety in opinions that result in great innovations. To stop your customers talking to your staff about ideas does not seem to make sense. When you take it a step further and stop politicians/civil servants talking to voters it makes even less sense. My understanding of the role of civil servants is quite limited, but I suspect we need to encourage them to debate topics in public using the same level of professionalism that they would use when discussing any given issue with their boss. Jeremy’s Whitehall Webb blog provides a nice example.
I was pleased to see that MP Tom Watson seems to get the idea; let’s hope his colleagues see the benefits of his conversations and start their own.
Exclusively positive blogs don’t work. People can tell when a PR machine has been working away on a statement, even when it’s cunningly disguised as a blog post – the tone of voice lacks authenticity. You need to let people talk about negatives, but at the same time empower them to turn them into positives.
Most complaints start with a small fault which only becomes a complaint through poor communication/management.
Forming tight guidelines on what you can and cannot say is never going to encourage people to blog, quite the opposite – and besides, lengthy policy documents never get read.
Just to confirm, The Guardian quotes me as saying that I believe that Civil Serf has been sacked. I actually said “If the civil servant has been identified and sacked it will be a real shame for the civil service”. In fact I suspect ’she’ may have pulled the blog to avoid being sacked. It’s mainly for this reason that I have not re-published the blog posts which google reader stored nicely for me.
Well she started an inevitable conversation which is never a bad thing. Looking back over the Civil Serf posts I don’t see any comments. I’m not sure if that’s because they were never enabled or they have not been cached. I would like to have read over peoples reactions to the blog. If there was no ability to add comments the blog was effectively just a scratching post.
As for setting back Government 2.0…I’m no fan of terminology 2.0 mainly because people seem to use such terms to describe things they don’t understand or can’t explain. I suspect we can learn a great deal about Government 2.0 and the issues that need to be tackled in order to make it a reality. Whatever that reality may be.
If you want to learn more about blogging in Corporates I can recommend reading Naked Conversation from Robert Scoble and Shel Israel. Its a good read.
http://groups.google.com/group/ccTiddly
Cheers
Simon
