Fork me on GitHub
Subscribe 4

Ticket #810 (fixed enhancement)

Improve unread forums tracking

  • Created: 2013-01-13 18:38:45
  • Reported by: Franz
  • Assigned to: quy
  • Milestone: 1.5.8
  • Component: performance
  • Priority: normal

As described in this topic, some of the unread forums tracking could be moved to the cookies - as is done for topics already.

This should then cause an index page performance boost for large forums.

History

Franz 2013-02-22 11:11:20

  • Milestone changed from 1.5.3 to 1.5.4.

adaur 2013-08-09 20:20:09

Maybe we should take care of this in 1.5.5?

Franz 2013-08-10 19:27:31

  • Milestone changed from 1.5.4 to 1.5.5.

quy 2013-10-20 18:55:52

Here is a proposed change. Instead of adding all new topics into the array, only forums with new topics will be added per forum. In this test case of 30,000 topics, potentially up to 30,000 can be added vs. adding maximum # of forums (20 in this case)...big difference.

Generation times are approximately the same but memory usage is less.
Before = [ Generated in 2.785 seconds, 9 queries executed - Memory usage: 3.97 MiB (Peak: 5.63 MiB) ]
After = [ Generated in 2.570 seconds, 9 queries executed - Memory usage: 709.87 KiB (Peak: 5.09 MiB) ]

find:

if (!$pun_user['is_guest'])
{
	$result = $db->query('SELECT t.forum_id, t.id, t.last_post FROM '.$db->prefix.'topics AS t INNER JOIN '.$db->prefix.'forums AS f ON f.id=t.forum_id LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=f.id AND fp.group_id='.$pun_user['g_id'].') WHERE (fp.read_forum IS NULL OR fp.read_forum=1) AND t.last_post>'.$pun_user['last_visit'].' AND t.moved_to IS NULL') or error('Unable to fetch new topics', __FILE__, __LINE__, $db->error());

	$new_topics = array();
	while ($cur_topic = $db->fetch_assoc($result))
		$new_topics[$cur_topic['forum_id']][$cur_topic['id']] = $cur_topic['last_post'];

	$tracked_topics = get_tracked_topics();
}

change:

if (!$pun_user['is_guest'])
{
	$tracked_topics = get_tracked_topics();

	$result = $db->query('SELECT t.forum_id, t.id, t.last_post, f.last_post AS forum_last_post FROM '.$db->prefix.'topics AS t INNER JOIN '.$db->prefix.'forums AS f ON f.id=t.forum_id LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=f.id AND fp.group_id='.$pun_user['g_id'].') WHERE (fp.read_forum IS NULL OR fp.read_forum=1) AND t.last_post>'.$pun_user['last_visit'].' AND t.moved_to IS NULL') or error('Unable to fetch new topics', __FILE__, __LINE__, $db->error());

	$new_topics = array();
	while ($cur_topic = $db->fetch_assoc($result))
	{
		if (!isset($new_topics[$cur_topic['forum_id']]) && (!isset($tracked_topics['topics'][$cur_topic['id']]) || $tracked_topics['topics'][$cur_topic['id']] < $cur_topic['last_post']) && (!isset($tracked_topics['forums'][$cur_topic['forum_id']]) || $tracked_topics['forums'][$cur_topic['forum_id']] < $cur_topic['forum_last_post']))
			$new_topics[$cur_topic['forum_id']] = 1;
	}
}

find:

	if (!$pun_user['is_guest'] && $cur_forum['last_post'] > $pun_user['last_visit'] && (empty($tracked_topics['forums'][$cur_forum['fid']]) || $cur_forum['last_post'] > $tracked_topics['forums'][$cur_forum['fid']]))
	{
		// There are new posts in this forum, but have we read all of them already?
		foreach ($new_topics[$cur_forum['fid']] as $check_topic_id => $check_last_post)
		{
			if ((empty($tracked_topics['topics'][$check_topic_id]) || $tracked_topics['topics'][$check_topic_id] < $check_last_post) && (empty($tracked_topics['forums'][$cur_forum['fid']]) || $tracked_topics['forums'][$cur_forum['fid']] < $check_last_post))
			{
				$item_status .= ' inew';
				$forum_field_new = '<span class="newtext">[ <a href="search.php?action=show_new&amp;fid='.$cur_forum['fid'].'">'.$lang_common['New posts'].'</a> ]</span>';
				$icon_type = 'icon icon-new';

				break;
			}
		}
	}

       
change:

	if (isset($new_topics[$cur_forum['fid']]))
	{
		$item_status .= ' inew';
		$forum_field_new = '<span class="newtext">[ <a href="search.php?action=show_new&amp;fid='.$cur_forum['fid'].'">'.$lang_common['New posts'].'</a> ]</span>';
		$icon_type = 'icon icon-new';
	}
Comment edited 2 times (Diff, Diff 2)

Franz 2013-11-14 00:20:39

  • Milestone changed from 1.5.5 to 1.5.6.

Thanks for the patch, Quy. That looks quite good already. We'll put in some extra testing for 1.5.6 so that this finally gets done.

quy 2013-11-21 15:40:03

  • Owner set to quy.

Revised code for performance gain when there are no tracked forums/topics:

if (!$pun_user['is_guest'])
{
	$result = $db->query('SELECT f.id, f.last_post FROM '.$db->prefix.'forums AS f LEFT JOIN '.$db->prefix.'forum_perms AS fp ON (fp.forum_id=f.id AND fp.group_id='.$pun_user['g_id'].') WHERE (fp.read_forum IS NULL OR fp.read_forum=1) AND f.last_post>'.$pun_user['last_visit']) or error('Unable to fetch forum list', __FILE__, __LINE__, $db->error());
	
	if ($db->num_rows($result))
	{
		$forums = $new_topics = array();
		$tracked_topics = get_tracked_topics();

		while ($cur_forum = $db->fetch_assoc($result))
		{
			if (!isset($tracked_topics['forums'][$cur_forum['id']]) || $tracked_topics['forums'][$cur_forum['id']] < $cur_forum['last_post'])
				$forums[$cur_forum['id']] = $cur_forum['last_post'];
		}	

		if (!empty($forums))
		{
			if (empty($tracked_topics['topics']))
				$new_topics = $forums;
			else
			{
				$result = $db->query('SELECT id, last_post, forum_id FROM '.$db->prefix.'topics WHERE forum_id IN('.implode(',', array_keys($forums)).') AND last_post>'.$pun_user['last_visit'].' AND moved_to IS NULL') or error('Unable to fetch new topics', __FILE__, __LINE__, $db->error());
	
				while ($cur_topic = $db->fetch_assoc($result))
				{
					if (!isset($new_topics[$cur_topic['forum_id']]) && (!isset($tracked_topics['forums'][$cur_topic['forum_id']]) || $tracked_topics['forums'][$cur_topic['forum_id']] < $forums[$cur_topic['forum_id']]) && (!isset($tracked_topics['topics'][$cur_topic['id']]) || $tracked_topics['topics'][$cur_topic['id']] < $cur_topic['last_post']))
						$new_topics[$cur_topic['forum_id']] = $forums[$cur_topic['forum_id']];
				}
			}
		}
	}
}
	if (isset($new_topics[$cur_forum['fid']]))
	{
		$item_status .= ' inew';
		$forum_field_new = '<span class="newtext">[ <a href="search.php?action=show_new&amp;fid='.$cur_forum['fid'].'">'.$lang_common['New posts'].'</a> ]</span>';
		$icon_type = 'icon icon-new';
	}
Comment edited 5 times (Diff, Diff 2, Diff 3, Diff 4, Diff 5)

Visman 2013-11-24 11:17:43

I have the strange question wink
In conditions and the _tracked_topics functions there are array cells of $tracked_topics['forums'], but to these elements where it isn't appropriated values.
Why they are necessary?

quy 2013-11-24 15:08:37

When a user clicks "mark forum as read", the forum ID and the current time is set in $tracked_topics['forums']. Did this answered your question?

Visman 2013-11-24 16:03:12

I see.

Franz 2014-01-08 23:14:46

  • Milestone changed from 1.5.6 to 1.5.7.

quy 2014-03-17 06:10:48

I revised the code once more (see above) and I am happy with it. @Franz okay to merge??

Franz 2014-03-18 20:56:07

Can you send a pull request, please? That makes it easier to review and we can just pull down the branch to test.

Thanks, you're awesome! smile

quy 2014-03-18 21:56:30

Commit bac3efd to fluxbb master

#810: Improve unread forums tracking

quy 2014-03-18 22:16:58

Done. Thanks.

Franz 2014-03-23 14:20:31

Commit 7630bb5 to fluxbb master

Merge pull request #100 from Quy/810

#810: Improve unread forums tracking

quy 2014-03-23 16:20:11

Commit d186a1a to fluxbb master

#810: Remove tabs; change order to match previously

quy 2014-03-23 16:22:44

  • Status changed from open to fixed.

Franz 2014-10-20 11:50:50

  • Milestone changed from 1.5.7 to 1.5.8.