Cleaning up Facebook “Friend” list

I created my Facebook account in 2007, when Facebook was slowly making its trend in this side of the world. And then I made the worst mistake – adding anyone.

Continue reading

Advertisements

[JIRA] Dev – Where are OSWorkflow Transition Screens?

Found a piece that isn’t well documented. But as usual, I’m explaining my steps and then the solution. It’s no real big deal anyway. 🙂

Here’s the background:
Mission was to write a Groovy script that returns all the transition screens used across all workflows of a given JIRA workflow scheme (for documentation purposes). This looked pretty easier, until a collection of ‘com.opensymphony.workflow.loader.ActionDescriptor’ came up without any linked JavaDocs. That looked like a dead end.

Then, with a little bit of googling I came across ActionDescriptor source code, which wasn’t helpful either. getView() method looked promising because ‘Transition View’ is a synonym used in JIRA UI for ‘Transition Screen’. But disappointingly it was just returning string ‘fieldscreen’ for every transition. Aw.. snap! 😦

Next attempt is called the ‘Bruteforce method’ (not java.lang.reflect.Method of course 😉 ).

Since I was too lazy to read across the ActionDescriptor’s source code:

// workflow is type of com.atlassian.jira.workflow.JiraWorkflow
// I'm not explaining all my code here
workflow.getAllActions().each {
  it.class.getDeclaredMethods().grep {
    it.name.matches( "^get.*" )
  }.each {
    log.debug ( it.name )
  }
}
getAutoExecute
getConditionalResults
getPreFunctions
getUnconditionalResult
getMetaAttributes
getValidators
getPostFunctions
getRestriction
getView
getName

It didn’t took much of time. With a bit of guessing I made my first attempt with getter method ‘getMetaAttributes()’. And it had the eye-catching piece I was looking for!

workflow.getAllActions().each {
  log.debug ( it.getMetaAttributes().toString() )
}
[opsbar-sequence:10, jira.description:, jira.fieldscreen.id:10000]
[opsbar-sequence:20, jira.description:, jira.fieldscreen.id:20000]
[opsbar-sequence:40, jira.description:, jira.fieldscreen.id:30000]
[opsbar-sequence:60, jira.description:, jira.fieldscreen.id:30000]

getMetaAttributes() returns a HashMap. So, finally…

def screens = [:]
workflow.getAllActions().each {
  def screenId = it.getMetaAttributes()["jira.fieldscreen.id"]
  if ( screenId != "" ) {
    def workflowAction = workflow.name + " - " + it.name
    def workflowActions = []
    if ( screens[screenId] ) workflowActions = screens[screenId]
    workflowActions.add ( workflowAction )
    screens[screenId] = workflowActions
  }
}
for ( e in screens ) {
  log.debug ( "Transition Screen: " + e.key )
  e.value.each {
    log.debug ( "  Workflow action: " + it )
  }
}

Nice! This HashMap (‘screens’) is all what I wanted. 🙂 Each transition screen, and then where it has been used.

So one last thing… I also wanted to have a one last peek into ActionDescriptor’s source code on GitHub. Only the writeXML() method gave an indication that metaAttributes is actually a HashMap. There was absolutely no indication that screen association information is kept as a ‘meta attribute‘.

Bottom line(s):

  1. If you are a JIRA developer and whenever you want to find screens associated to a workflow, don’t use getView(). Use getMetaAttributes() and then look for the attribute with key “jira.fieldscreen.id”. In other words, transition screen is stored as a meta attribute in each transition. Screen is referred by its numeric ID as a string.
  2. Since OpenSymphony doesn’t seem to go away from JIRA, and some already claim that OpenSymphony is dead, it would be worthwhile if Atlassian maintains their own repository of OSWorkflow docs and special remarks.

JIRA project migration with Google Spreadsheets

JIRA to JIRA project migration — most of the time this has been a headache when it comes to customer negotiations and compromises.

In my experience, usually it starts with a discussion round to identify what customer exactly needs. Some customers expect everything, including issue change history, comments, votes, etc. In this case we have to use the XML project import method. With status, resolution, priorities and custom field mapping this is a headache. In the near-perfect JIRA project migrations I’ve performed, I remember I used Groovy scripts to merge field values, convert additional issue types to labels, and etcetera. If you choose XML import, it needs a lot of technical work and beforehand testing when you have a demanding customer and a JIRA instance that isn’t expected to change much.

The other type of customers are simple-living minimalist people. 😉 They keep things simple, needs are simple and they just need the contents to be brought over. They are okay with basic things such as issue summary, description, type and status.

The method I’m going to discuss today is for the customers who lie in-between the two types of customers above. In my particular case I met a customer who needs comments and one of their custom fields to be brought over. File attachments and issue change history were not necessary for them. In this case downloading JIRA issues from Issue Navigator into Excel and using it as a CSV source won’t do the perfect import.

This led me to develop a custom Google Spreadsheets script. It uses JIRA REST API and fetches project issues into the spreadsheet. Then, it can be downloaded as CSV and imported into JIRA. Here’s a reduced version of the code I wrote – just to demonstrate:

var jql = encodeURIComponent("project = ABC ORDER BY key ASC");
var rows = [];
var s = 0;
while (true) {
  var response = fetchJIRA (baseurl, "/rest/api/2/search?jql=" + jql + "&maxResults=200&fields=*all%2Ccomment&startAt=" + s);
  // fetchJIRA () is a custom function I wrote 
  // to call JIRA REST API and return JSON.
  // It is not explained in this article.
  var max = response.maxResults;
  var c = response.issues.length;
  var t = response.total;
  if (c == 0) break;

  response.issues.forEach (function (i) {
    var row = [];
    row.push (i.key);
    row.push (i.fields.summary);
    // .. Include other fields
    rows.push (row);
    s = response.startAt + c;
  });
}
rows.unshift (["Key", "Summary", ...]);

var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
var sheet = spreadsheet.insertSheet("Issues");
var iR = sheet.getRange(1, 1, rows.length, rows[0].length);
iR.setValues(rows);

Actually it does more!

  • Status/ issue type/ resolution/ user account/ select-list custom field value mapping:
    Using a function like this maps statuses on-the-fly fetching, so we don’t have to worry about creating a workflow.

    function statusMapper (param) {
    var mapping = {
      "Open":"Backlog",
      "Done":"Closed",
      "Resolved":"Closed",
      "Validate":"In Verification",
      "Planned":"Defined"
    };
    var ret = mapping[param];
    if (ret == undefined) ret = param;
    return ret;
    }
    row.push (statusMapper (i.fields.status.name));
    row.push (typeMapper (i.fields.issuetype.name));
    row.push (i.assignee ? userMapper (i.assignee.name) : "shaakunthala");
    // ^^ Handy if target project doesn't allow unassigned issues.
    // It maps user accounts on-the-fly, 
    // while assigning all unassigned issue to a default user account.
  • Linking parent and sub-task on-the-fly. The script will do what’s asked in the Atlassian’s official documentation. After fetching issues, ParentIDs will be added using VLOOKUP spreadsheet formula applied by the script.
    var issueid_col = sheet.getLastColumn();
    var newCol = issueid_col + 1;
    var parentkey_cell1 = sheet.getRange(2, issueid_col - 1, 1, 1).getA1Notation();
    var formula ="=IFERROR(VLOOKUP(" + parentkey_cell1 + "," + issuesRange.getA1Notation().replace (/[0-9]/g, "") + "," + issueid_col + ",FALSE), \"\")";
    sheet.getRange(1, newCol, 1, 1).setValue("Parent ID");
    sheet.getRange(2, newCol, 1, 1).setFormula(formula)
         .copyTo(sheet.getRange(3, newCol, sheet.getLastRow() - 2, 1));
  • Mapping additional custom fields to labels. This is useful when you want to avoid creating any custom fields.
    if (i.fields.customfield_30000 != null)
      labels_array.push (i.fields.customfield_30000.value);
  • Comments, Affects Versions, Fix Versions, Labels, etc. can be fetched into different sheets of the same spreadsheet to be imported separately. The reason for this is to avoid large and much complicated CSV files that aren’t much human-readable.
    • In this case, still, target project doesn’t have to be empty. If target project is different, relationship between multiple CSV imports can be handled with issue key inserted to a text custom field as a reference ID.

So, here’s the pros and cons.

Pros:

  1. What’s being transferred over are in human-readable format in the spreadsheet. It may be useful reference. This gives you a better way to verify any mapping and additional labeling you have done.
  2. Google Spreadsheets will take care of the CSV syntax and escaping. You don’t have to worry a thing about CSV errors. Just code what needs to sit in each column.
  3. Less time consuming compared to XML import, but brings over more data compared to minimal CSV import.

Cons:

  1. Limitations of Google Spreadsheets: Google will terminate the script if it runs for more than 6 minutes. This can happen with larger projects. In this case you can use a Python script instead. But you’ll need to write additional code to map parents and sub-tasks.

Reference: 

Official documentation by Atlassian: https://confluence.atlassian.com/adminjiraserver071/importing-data-from-csv-802592885.html

Notes:

There could be third-party add-ons that can perform hassle-free project migrations. If you can purchase such an add-on, this kind of approaches may not be necessary.

There is a method to import JSON instead of CSV. This syntax appears to be different from the issue object returned by the REST API. Therefore you’ll still need a custom script to perform JSON import.

Wunderlist Today wallpaper for Ubuntu

Wunderlist again. Currently this is the only Microsoft thing that I use – apart of Windows servers involved in my DevOps job.

Wunderlist hasn’t been treating nice for Ubuntu recently. So I thought about working on something that works for Ubuntu. Then here I got an idea of a very small app that will embed your day action items into the desktop wallpaper. Since I’m not very much into software development, this as a pilot project will make me comfortable for building something better for Ubuntu.

To be honest, first I was planning to do some browser DOM hacking and figure out how it works. Soon after, I found out that there’s a public API for Wunderlist (awesome, isn’t it?), and that can avoid the need of DOM hacking. So, that’s how wunderlist-ubu-wallpaper started as my pilot project.

It is still very young, and I’m sure my code needs a little cleanup as well. It’s just a 150 line Python script – not at all a big deal. Readme file has all the information you need to set it up on your Ubuntu. Run it and see what happens to your Ubuntu wallpaper. Isn’t that awesome?

14 years back when I was a kid, I was just making little Windows executables (I was a Windows user back then) with Visual Basic. After that I did programming to a some level when I was a systems administrator, but I was never permitted to release my work to the public for obvious reasons. After that, here comes my very first little code contribution to the world of Open Source.

By the way, my all other action items are overdue! 😳

[JIRA] Delete projects from XML backup using Python

[Experimental]

If you are a JIRA system administrator, have you ever come across the requirement of partially exporting JIRA into XML – probably for importing individual projects into another JIRA instance?

If yes, this might work for you. However, my work is incomplete and experimental. This worked with a small JIRA instance (100 issues), and failed with a huge JIRA instance (1 million issues).

Here’s the Python script I wrote for deleting projects from JIRA XML backup:

#!/usr/bin/python

from lxml import etree

#xp = etree.XMLParser(encoding='utf-8', recover=True)
#doc = etree.parse('entities.xml', xp)
doc = etree.parse ('entities.xml')

# Array of project keys
plist = ['ABC', 'DEF', 'GHI']

for x in plist:
    for p in doc.xpath ("//Project[@key!=\'" + x + "\']"):
        #p.get entities and delete
        pi = p.get("id")
        for i in doc.xpath ("//Issue[@project=\'" + pi + "\']"):
            ii = i.get("id")
            for cfv in doc.xpath ("//CustomFieldValue[@issue=\'" + ii + "\']"):
                cfv.getparent().remove(cfv)
            for cg in doc.xpath ("//ChangeGroup[@issue=\'" + ii + "\']"):
                cgi = cg.get("id")
                for ci in doc.xpath ("//ChangeItem[@group=\'" + cgi + "\']"):
                    ci.getparent().remove(ci)
                cg.getparent().remove(cg)
            for fa in doc.xpath ("//FileAttachment[@issue=\'" + ii + "\']"):
                fa.getparent().remove(fa)
            for il in doc.xpath ("//IssueLink[@source=\'" + ii + "\']"):
                il.getparent().remove(il)
            for il in doc.xpath ("//IssueLink[@destination=\'" + ii + "\']"):
                il.getparent().remove(il)
            for na in doc.xpath ("//NodeAssociation[@sourceNodeEntity=\'Issue\' and @sourceNodeId=\'" + ii + "\']"):
                na.getparent().remove(na)
            for wl in doc.xpath ("//Worklog[@issue=\'" + ii + "\']"):
                wl.getparent().remove(wl)
            for c in doc.xpath ("//Action[@issue=\'" + ii + "\']"):
                c.getparent().remove()
            i.getparent().remove(i)
        for na in doc.xpath ("//NodeAssociation[@sourceNodeEntity=\'Project\' and @sourceNodeId=\'" + pi + "\']"):
            na.getparent().remove(na)
        for c in doc.xpath ("//Component[@project=\'" + pi + "\']"):
            c.getparent().remove(c)
        for v in doc.xpath ("//Version[@project=\'" + pi + "\']"):
            v.getparent().remove(v)
        for a in doc.xpath ("//ProjectRoleActor[@pid=\'" + pi + "\']"):
            a.getparent().remove(a)
        for pk in doc.xpath ("ProjectKey[@projectId=\'" + pi + "\']"):
            pk.getparent().remove(pk)
        p.getparent().remove(p)

f = open ('entities.new.xml', 'w')
f.write (etree.tostring(doc, pretty_print=True, xml_declaration=True))
f.close()

Before you run this, you need to unzip the JIRA XML backup which is a zip archive containing two or three XML files. Put the script file in the same directory where you have the unzipped XML files and run it. It will produce output into file ‘entities.new.xml’. Once completed, replace ‘entities.xml’ by ‘entities.new.xml’, remove the Python script and re-pack the folder into a new zip file. Now it is ready to be imported into another JIRA instance.

Please note that this is incomplete and I cannot guarantee 100% success. You can easily follow the structure of this code and improve it by yourself.

Password trainer software

I screwed it up again.

I logged into one of my e-mail accounts from a shared computer owned by someone else. This is something I never do (unless there’s a top urgency) because I fear keyloggers. So now, time has come to change the passwords. Next factor of two factor authentication will keep me protected, but still need to fix the first factor.

I know one silly things people would do, especially when password expiry is in place. The first password will be ‘password‘, then after two months it will become ‘passwore‘, next ‘passworf‘, ‘passworg‘, and so on. But this isn’t the time for that game. By any chance if my password has been key-logged, it’s all in bare naked form.

And that’s how I had to compose a new set of passwords. That wasn’t easy. I’m not going to tell how they were constructed but they are lengthy and look like garbage. Now, I need to train my fingers for the new passwords so I won’t forget them easily, and because enough speed will hide it from shoulder surfers. (I never write them down on sticky notes)

So that’s how this idea came along: A JavaScript based password trainer. It’s very simple and all you need is to use your web browser (Chrome/ Firefox).

First, go to the URL ‘about:blank‘. This is important – do not try this while you are in any web page that you can’t trust.

Open your web browser’s developer tools (F12 for Firefox & Chrome), or may be Firebug, and switch to the Console.

Copy-paste the following JS code there, and replace the text ‘mypwd’ in the following JS code with your complex, garbage-like password. Execute it!

If you successfully type in your password more than 25 times and keep success rate above 95%, it will stop looping through.

success = 0;
fail = 0;
while (true) {
  password = "mypwd";
  if (success > 25 && (success/(success+fail)) > 0.95) {
    break;
  }
  match = prompt ('Please enter password\nSuccess: ' + success + '\nFail: ' + fail);
  if (match == password) {
    success = success + 1;
  } else {
    fail = fail + 1;
  }
}
alert ('Your fingers are now trained!');

I wish if I could fix something like this into my cheap acoustic guitar. 🙂

[JIRA] Bulk deleting pojects like a boss!

These days I’m working on a JIRA project which may require working with XML backups and project import tool. The JIRA instance I’m currently working on has over one million issues in thousands of projects (including defunct) – that collectively make the XML backup gigantic.

Have you ever come across the situation where you want to delete some of the projects in JIRA to reduce the size of XML backup? Well, here’s a quick script that I wrote. However, it requires the Adaptavist ScriptRunner add-on which is no longer free. 😦

Continue reading