- Added pdfinfo support to determine number of PDF pages (requires custom binary)
- Automatic downloading and installation of customized Xpdf tools from zotero.org
Other changes:
- Various API Zotero.Fulltext API changes
- Prevent buggy automatic prefpane sizing
- New 'refresh' Notifier action to just reselect the currently selected item without sending a 'modify' trigger
- Zotero.File.putContents(file, str) -- can only handle ASCII text
- Zotero.isLinux property
- New Actions menu option, "Report Errors...", to send chrome errors from the Error Console to the Zotero server via the error reporting wizard -- no more Advanced Instructions from the Reporting Bugs page
- Disabled unresponsive script indicator during schema upgrades, since that seems to be what caused problems during the Beta 4 upgrade
- Added repair steps for interrupted Beta 4 upgrades
- Speed up b4.r2 itemNotes repair step, which may have been what was causing the unresponsive script warning on large libraries
- Moved some Zotero.UnresponsiveScriptIndicator.enable() calls into finally{} blocks to make sure they get triggered
- New method Zotero.getSystemInfo() to get some info on the environment (platform, version, app, app version, locale) as a string for inclusion in bug reports
- Added public property skipBackup to Zotero.DB so a startup error can cancel the shutdown backup
Closes#321, Notification window to show when rebuilding cache
Closes#560, Method for inserting bibliography/export format into text box on current page
Dragging items to external text apps is also possible now (the one oddity being that the format has to be set at the beginning of the drag session, so if the window's current page matches a site-specific setting, the dragged references will use that format even the if the text is dropped into an external app or another window)
Added proceedingsTitle with base field publicationTitle for now, since the hierarchical item type structure will require both (the conference itself will be an event)
Copied any conferenceName value to proceedingsTitle
- Massive optimization of data layer -- with ~11,000-item test library on a Mac Pro, decreased initial Zotero pane loading from several minutes to ~10 seconds. This included some small API changes and new methods (e.g. Items.cacheFiles()) in the data layer, but most of it was changing the way loading and caching of data worked internally.
- Moved unique itemData values out to separate itemDataValues table for better normalization
- Updated itemTreeView.sort() to be able to sort a single row into the items list for performance reasons -- itemTreeView.notify() now only sorts a single row when possible (and sometimes doesn't need to sort anything). This should make general interface use dramatically less sluggish with large libraries.
- Consolidated purging on item deletes, which should speed up multi-item deletes quite a bit -- clients should use Items.erase() instead of Item.erase(), since the former calls the new Items.purge() method (which calls the various other purge() methods) automatically
- Notifier no longer throws errors in notify() callbacks and instead just logs them to the Error Console -- this way a misbehaving utility (or Zotero itself) won't keep other observers from receiving change notifications
- Better handling of database corruption -- if an SQL query throws a file corruption error, Zotero adds a marker file to the storage directory and displays a message prompting the user to restart to attempt auto-repair--and, most importantly, no longer copies the corrupt file over the last backup.
- A "Loading items list..." message appears over the items list (at least, sometimes) while data is loading -- useful for large libraries, but may need to be fine-tuned to not be annoying for smaller ones.
- Note titles are now cached in itemNoteTitles table
- orderIndex values are no longer consolidated when removing items from collections -- it just leaves gaps
- Fixed shameful bug in getRandomID() that could result in an item with itemID 0, which wouldn't display correctly and would be impossible to remove
- Fixed autocomplete and search for new location of 'title' field
- Added proper multipart date support for type-specific 'date' fields
- Made the pre-modification array passed to Notifier observers on item updates actually be pre-modification
- New method Zotero.ItemFields.isFieldOfBase(field, baseField) -- for example, isFieldOfBase('label', 'publisher') returns true, as does isFieldOfBase('publisher', 'publisher')
- Restored ability to drag child items in collections into top-level items in those collections
- Disabled unresponsive script message when opening Zotero pane (necessary for large libraries, or at least was before the optimizations)
- Collections in background windows didn't update on item changes
- Modifying an item would cause it to appear incorrectly in other collections in background windows
- Fixed an error when dragging, hovering to open, and dropping a note or attachment on another item
- Removed deprecated Notifier methods registerCollectionObserver(), registerItemObserver(), unregisterCollectionObserver(), and unregisterItemObserver()
- Loading of Zotero core object can be cancelled on error with Zotero.skipLoading
- Removed old disabled DebugLogger code
- New method Zotero.log(message, type, sourceName, sourceLine, lineNumber, columnNumber, category) to log to Error Console -- wrapper for nsIConsoleService.logMessage(nsIScriptError)
- New method Zotero.getErrors(), currently unused, to return array of error strings that have occurred since startup, excluding CSS and content JS errors -- will enable an upcoming Talkback-like feature
- Fixed some JS strict warnings in Zotero.Date.strToMultipart()
- All remaining fields in the items table exists in all items and are non-user-editable
- Simplified some data access code (e.g. removed Item.isEditableField())
- 'title' is now a base field used in case (Case Name), statute (nameOfAct) and e-mail (Subject)
Reengineered parts of the data layer for better performance
- Various recent changes, including the 'title' change above and base field mapping in Item.getField(), had a negative effect on performance. This should help. In particular, itemData values are now loaded in in bulk by Items._load() (via Items.get()) rather than on-demand, the sort process in itemTreeView caches values while sorting, and ItemFields.getFieldIDFromTypeAndBase() is faster.
Addresses #346, mapping for new item types
Closes#227, Indent nested collections in search drop-down
Addresses #528, Make search condition drop-down menu less unwieldy
- Created new distinct fields for differently labeled fields
- Mapped lots of fields to base fields
- Made base field search conditions search type-specific fields as well
- Removed type-specific fields that are based on base fields not show up in search conditions drop-down
- Added a tooltip when hovering over a condition in the search conditions drop-down that shows the fields it searches (when there's more than one)
- Moved search dialog CSS to separate file
Abstract displays in metadata pane as a cropped one-line field by default; clicking the 'Abstract' label toggles between the cropped field and an expanded view
Some problems with import/export: https://www.zotero.org/trac/ticket/537
Refs #537
Addresss #352, Make sure data layer doesn't allow bad data via the API
Access date field is now human-friendly. Also enforcing SQL date form for the field in the DB and discarding bad data passed via setField().
Closes#453, Check if any fields will actually be discarded on item type change before giving warning
Refs #530, Add base field conversion to translation level
Added mechanism for linking item type fields via base fields, e.g. publisher => label in audioRecording
New methods:
Item.getFieldsNotInType(itemTypeID, allowBaseConversion)
ItemFields.getLocalizedString(itemTypeID, field)
ItemFields.isBaseField(fieldID)
ItemFields.getFieldIDFromTypeAndBase(itemType, baseField)
ItemFields.getBaseIDFromTypeAndField(itemType, typeField)
ItemFields.getTypeFieldsFromBase(baseField)
Currently only the publisher fields are mapped -- I need more feedback on #346 before I implement the others (specifically on whether or not all these sorts of fields should be done as distinct fields or whether some should just be localized strings (in which case they'll autocomplete but not show up separately as search conditions))
Also added 'university' as distinct publisher field for thesis
Values of equivalent fields are now preserved when switching between item types (e.g. the 'studio' value becomes the 'label' value when switching between videoRecording and audioRecording), and the pop-up is much smarter--it will only prompt you if fields will in fact be lost, and it will list the fields that would be deleted.
Not finished:
- Searching for base fields doesn't yet search the type-specific fields, as Elena requested
- import/export/bib should be updated to use the ItemFields base conversion methods where appropriate -- data coming from the 'publisher' field from translators, for example, should be put into the appropriate type-specific field.
- Automatic tags now appear in orange; tooltip says either "User-added tag" or "Automatically added tag"
- New menu in tag selector to toggle automatic tags
- User and automatic tags are combined in tag selector, so renaming/deleting a tag will affect both user and automatic, regardless of view mode
- Editing a tag makes it a user tag, as does adding an identical user tag to an item (rather than creating a second one)
- ingester/export will need to be adjusted to add automatic tags
Changed:
Item.addTag(tag) => addTag(tag, type)
Item.getTags() - now returns 'id', 'tag', 'type'
Item.toArray() - tags now include 'type' property (from Item.getTags())
Tags.getID(tag) => getID(tag, type)
Tags.getAll() => getAll([types]) - types is an optional array of tagTypes to fetch; now returns objects with 'tag' and 'type' properties
Tags.getAllWithinSearch(search) => Tags.getAllWithinSearch(search, [types]) - now returns object with 'tag'/'type'
Added:
Tags.get(tagID) - returns object with 'tag' and 'type' properties
Tags.getIDs(tag) - returns all tagIDs for this tag (of all types)
Tags.getType(tag) - returns array of tag types matching given tag
For type property, 0 == user, 1 == automatic
closes#441, In-page annotations of snapshots
testers: please check pages containing both annotations and highlights to make sure everything works correctly
Added minVersion and maxVersion times to existing scrapers, setting 1.0.0b3.r1 as minVersion for any >4096 characters; these could theoretically now be added back to the repository without problems, but there's not really much reason to test that theory at the moment
- Item.setField() stores dates in a multipart format beginning with an SQL date followed by the user's entry, so "November 3, 2006" becomes "2006-11-03 November 3, 2006" -- date field entries are parsed with Zotero.Date.strToDate() if not already in multipart format
- Item.getField() returns just the user part unless passed the new second parameter, _unformatted_, which returns the field directly from DB without processing (e.g. the full multipart string)
- Added SQLite triggers on the itemData table to enforce multipart format even if the table is modified outside the API
- Migration step to update existing dates
- Indicator next to date field to show what we've parsed and a tooltip over the date field to show the SQL date -- though I'm not sure how well the abbreviation part will localize (i.e. can you abbreviate 'month' in Chinese?)
One obvious problem is how to handle date ranges when sorting or searching, which may end up rendering this whole method fairly useless (though I guess the multipart format could begin with two SQL dates instead of just one, at the cost of some storage space...).
Other changes:
- Utilities.lpad() handling for undefined value parameter
- new Zotero.Date methods: strToMultipart(), isMultipart(), multipartToSQL(), multipartToStr(), isSQLDate(), sqlHasYear(), sqlHasMonth, sqlHasDay getLocaleDateOrder() (the last one unused for now)
- try/catch around manual itemData INSERT execute() statements in Item.save()
- Catch errors trying to display missing files and display message to user
- Switch to persistent descriptors rather than relative paths for attachment paths -- this will fix attachments on networked drives (which, at least on Windows, were not working and apparently in some cases breaking entire Zotero installs), but since WebBrowserPersist.saveDocument() is asynchronous and file.persistentDescriptor can't be set on Macs before the file exists, Attachments.importFromDocument() no longer returns the id of the new attachment, so translate.js had to be changed accordingly
- Try to convert relative descriptors to persistent ones with migration step (and later on access, if persistent fails)
- Added Zotero.WebProgressFinishListener(onDone)
Next step would be to throw up a file dialog to let the user locate the missing file
If anyone's out there, now would be the time to test this.
Missing icons for the new types -- I could try to get some of those now, or we could just launch. Maybe we'll just launch. But really, might want to test this. Just sayin'.
Including in the DB, which it turns out isn't really all that bad (thanks, among other things, to SQLite's ability to DROP tables within transactions without autocommitting (which MySQL can't do))
