track tickets/replies with thread-index

[dr_patso]

I'd like to discuss the problems or potentials with tracking tickets and replies using the message-ID vs containing the ticket number in the subject [#xxx=xxx=xxxx]

Couldn't We track replies loading the message ID into the hesk database and checking the "In reply to" message ID to track any replies?

This sparks my curiosity I have already implemented 3 cron jobs (1 for each user) where cron jobs call the imap_user.php and it calls pipe_functions_user.php that loads the email pasted into the IMAP account from outlook and assigns to the user. (each user has their own folder and cron job checking that folder)

[dr_patso]

is it because message IDs are not necessarily unique? Does the same user have the same message IDs?

[Klemen]

Message ID is generated by the server, not by HESK. HESK doesn't know what the message ID is of for example a ticket reply notification email, so it can't rely on using message ID.

[dr_patso]

So this could work if all tickets were created from e-mail piping? Of course the hesk admin would have to implement the code themselves (I understand hesk does not currently track message-id and probably does not have plans to) to find and load the message ID into the database with the ticket when it's created and check the in-reply-to message-id against the DB for existing ticket to add reply to.

[Klemen]

It wouldn't work.

When you respond to a ticket an email notification is sent to the customer. You DO NOT know message ID of this email notification so if the client responds to this email HESK won't know which ticket it belongs to.

[dr_patso]

I think this could work for me as we do not use hesk conventionally. I do not send any email out of hesk all notify_customer code is disabled. Hesk is transparent to the customers/clients/end users and purely used for KB and tracking.

Right now, I've added a new feature instead of manually creating tickets. I copy and paste outlook emails into an IMAP account / folder and hesk grabs them. I add replies I've already sent in outlook as needed into hesk manually with absolutely no notification to customer, I categorize, add time spent and mark resolved. It's the only way I can see using hesk until mutliple e-mails/requesters are supported (To: and CC:), multiple users need resolution on most of our cases. Also we send embedded screenshots a lot for instruction.

I Know we should be using a ticket system with outlook integration but to be honest the company is cheap and the imap folder hesk is checking works better than any outlook integration I've seen. It's all garbage! Hesk is the best / easiest ticketing system I've seen with KB and the team is not using it conventionally at all nfortunately

I don't think I could get anyone on the team comfortable with using hesk solely and moving away from outlook even if the embedded images / multiple recipients features are added in the future.

My thought is that if I can track the message ID when email is converted to ticket any subsequent emails IN REPLY TO original message ID get tracked as a reply.. So all I'm doing all day is dragging new tickets(email requests) and replies into hesk IMAP folder in outlook and following up with unresolved cases or marking tickets resolved at the end of the day. I really cannot see a better way when your team needs the email functions of outlook. The replies in hesk are limited to basic text and response to 1 user. It's as if Hesk is designed to support 1 user at a time only, perfect for a lot of end user support IT teams.

[Klemen]

HESK was indeed designed for end user support. Adding multi user functionality would probably complicate things and move HESK away from it's design philosophy (a simple alternative to complex systems).

If you believe tracking message ID would be beneficial you are of course free to edit your help desk to do just that. But in official version this is not planned as it isn't doable in the usual HESK usage sense.

[dr_patso]

understood,

I've thought about using the first 30 chars of thread-index in the e-mail headers in a new field in the hesk_tickets table, and checking that for a match to add replys vs creating new ticket. It appears I could borrow a lot of the logic from the code to check the subject for the trackid.

I've tested outlook/iosmailapp/gmail and a few others to find that when replying they retain the original messages 30 characters for the thread-index and add some hex/numbers to the first original 30 characters.

[dr_patso]

I add some logic to add first 30 chars of the thread index into the database. I hope I can use this to add replies since I am manually adding emails to email account for hesk piping via IMAP from outlook.

Why I chose thread-index: I use outlook, every message I've tested has a thread index. I believe outlook already does logic on this header via message-ids.. The problem wiht tracking the message id is at the end of a very long chain where your team is first added, there may be 10-15 message-ids in the "In reference to:" header. the in-reply-to: header only has the most recent original message, so it will only work for the first reply. The Thread-Index starts on the first email in the chain at 30 characters. Every subsequent message in the chain adds 6 bytes to the original thread-index.. This is perfect until someone changes the subject.. which is something you just have to deal with.. who knows maybe they changed the subject because it really should be a whole new ticket? So by checking the first 30 characters which appear to be extremely unique, you can determine that the message is part of that chain. I will be testing this but so far it appears to work reliably.

I have tested some other email clients, gmail, outlookweb(hotmail),yahoo and they seem to also have a 30 char thread-index

use at own risk, I wanted to share my efforts

In phymyadmin I added one basic column at the end of the hesk_tickets table labeled "threadindex"

varchar (70) utf8_unicode_ci


/inc/mail/mime_parser.php 2474-2485 (below the subject encoding code)

Code: Select all

		if(IsSet($message['Headers']['thread-index:']))
		{
			if(IsSet($message['DecodedHeaders']['thread-index:'])
			&& count($message['DecodedHeaders']['thread-index:']) == 1
			&& count($message['DecodedHeaders']['thread-index:'][0]) == 1)
			{
				$results['Threadindex'] = $message['DecodedHeaders']['thread-index:'][0][0]['Value'];
				$results['ThreadindexEncoding'] = strtolower($message['DecodedHeaders']['thread-index:'][0][0]['Encoding']);
			}
			else
				$results['Threadindex'] = $message['Headers']['thread-index:'];
		}

/inc/mail/email_parser.php line 289-291 (right below subject encoding)

Code: Select all

	// Thread-Index and Thread-Index encoding
	$r["threadindex"] = $result["Threadindex"];
	$r["threadindex_encoding"] = isset($result["ThreadindexEncoding"]) ? strtoupper($result["ThreadindexEncoding"]) : "";

/inc/pipe_functions.inc.php line 80-86

Code: Select all

	// Process email Thread-Index, convert to UTF-8, set to "" if none set
	$tmpvar['threadindex'] = isset($results['threadindex']) ? $results['threadindex'] : $hesklang['threadindex'];
	if ( ! empty($results['threadindex_encoding']) )
	{
		$tmpvar['threadindex'] = hesk_encodeUTF8($tmpvar['threadindex'], $results['threadindex_encoding']);
	}
	$tmpvar['threadindex'] = hesk_input($tmpvar['threadindex'],'','',1,30) or $tmpvar['threadindex'] = $hesklang['threadindex'];

/inc/posting_functions.inc.php line 86, 125, 144

86 below 'custom20'

Code: Select all

		`threadindex`

125 below 'custom20'

Code: Select all

		'".hesk_dbEscape($ticket['threadindex'])."'

144 below 'lastchange'

Code: Select all

	'threadindex'	=> $ticket['threadindex'],

Lastly

/language/en/text.php line 1023

Code: Select all

$hesklang['threadindex']=''; // Default Thread-Index for piped tickets without threadindex

^ this is so if no thread-index is present, the field will be null.

The end result is the first 30 chars of the thread-index being loaded into the hesk_tickets table.. It does not seem to affect other hesk functionality, I was a little worried about what adding a column to hesk_tickets would do.

[dr_patso]

Ok, I have the code in the pipe_functions.inc.php to track reply if the thread-index matches with a ticket in the database..

I added it right below the if then logic to check for a tracking id in the subject around line 147. I discarded the logic to check for matching email address since you may track your own reply, or customers by adding it to the IMAP account via outlook that hesk is conneting to.

line 147 pipe_functions.inc.php

Code: Select all

	// If threadindex matches one in the database set tracking ID
	$resolute = hesk_dbQuery("SELECT * FROM `".hesk_dbEscape($hesk_settings['db_pfix'])."tickets` WHERE `threadindex`='".hesk_dbEscape($tmpvar['threadindex'])."' LIMIT 1");
	if (hesk_dbNumRows($resolute))
	{
		
		$ticket = hesk_dbFetchAssoc($resolute);
		$tmpvar['trackid'] = $ticket['trackid'];
		
	
		if ($ticket['locked'])
	    {
	        	$tmpvar['trackid'] = '';
	    }
	    
	}
	

add the above code below the following (this logic above doesn't run unless you find a tracking ID in the subject)

Code: Select all

	        // Is this ticket locked? Force create a new one if it is
	        if ($ticket['locked'])
	        {
	        	$tmpvar['trackid'] = '';
	        }
	    }
	    else
	    {
	    	$tmpvar['trackid'] = '';
	    }
	}

I may need to add some logic to check for an empty tracking ID first to keep the logic to track replies via "[#trackid]" in the subject functioning, as it stands it WILL break that feature.

[dr_patso]

This is better. [#trackid] in the email subject will override the code to check threadindex.

/inc/pipe_functions.inc.php

Code: Select all

	// If thread-index matches one in the database set tracking ID for reply
	if ( empty($tmpvar['trackid']) )		
	{
		$resolute = hesk_dbQuery("SELECT * FROM `".hesk_dbEscape($hesk_settings['db_pfix'])."tickets` WHERE `threadindex`='".hesk_dbEscape($tmpvar['threadindex'])."' LIMIT 1");
		if (hesk_dbNumRows($resolute))
		{
		
			$ticket = hesk_dbFetchAssoc($resolute);
			$tmpvar['trackid'] = $ticket['trackid'];
		
	
			if ($ticket['locked'])
			{
					$tmpvar['trackid'] = '';
			}
		}
	}

just add right above this code.

Code: Select all

	// If tracking ID is empty, generate a new one