With icons to identify collections and searches
Also:
- `savedSearch` search condition in general
- Clean up some search window code
- Reorganize search tests
While trying to get translation and citing working with asynchronously
generated data, we realized that drag-and-drop support was going to
be...problematic. Firefox only supports synchronous methods for
providing drag data (unlike, it seems, the DataTransferItem interface
supported by Chrome), which means that we'd need to preload all relevant
data on item selection (bounded by export.quickCopy.dragLimit) and keep
the translate/cite methods synchronous (or maintain two separate
versions).
What we're trying instead is doing what I said in #518 we weren't going
to do: loading most object data on startup and leaving many more
functions synchronous. Essentially, this takes the various load*()
methods described in #518, moves them to startup, and makes them operate
on entire libraries rather than individual objects.
The obvious downside here (other than undoing much of the work of the
last many months) is that it increases startup time, potentially quite a
lot for larger libraries. On my laptop, with a 3,000-item library, this
adds about 3 seconds to startup time. I haven't yet tested with larger
libraries. But I'm hoping that we can optimize this further to reduce
that delay. Among other things, this is loading data for all libraries,
when it should be able to load data only for the library being viewed.
But this is also fundamentally just doing some SELECT queries and
storing the results, so it really shouldn't need to be that slow (though
performance may be bounded a bit here by XPCOM overhead).
If we can make this fast enough, it means that third-party plugins
should be able to remain much closer to their current designs. (Some
things, including saving, will still need to be made asynchronous.)
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
Get rid of data_access.js, at long last. Existing calls to
Zotero.getCollections() will need to be replaced with
Zotero.Collections.getByLibrary() or .getByParent().
Also removes Zotero.Collection::getCollections(), which is redundant
with Zotero.Collections.getByLibrary(), and Zotero.Collections.add().
The latter didn't didn't include a libraryID anyway, so code might as
well just use 'new Zotero.Collection' instead.
- Fixes some saving and erasing issues with collections and searches
- Adds Zotero.DataObject::eraseTx() to automatically start transaction,
and have .erase() log a warning like .save()
- Adds createUnsavedDataObject() and createDataObject() helper functions
for tests
0 allowed for some accidental behavior due to old code expecting NULL,
and it prevented easy checks (``if (!libraryID)``) for a passed
libraryID. Code now uses Zotero.Libraries.userLibraryID instead of a
hard-coded value (except in schema.js). Functions can still make
libraryID optional, but they should then use
Zotero.Libraries.userLibraryID if that's to mean the user library.
There might be some code that still expects 0 that I missed.
- Protocol handler extensions can now handle promises and can also make
data available as it's ready instead of all at once (e.g., reports now
output one entry at a time)
- zotero:// URL syntaxes are now more consistent and closer to the web
API (old URLs should work, but some may currently be broken)
Also:
- Code to generate server API, currently available for testing via
zotero://data URLs but eventually moving to HTTP -- zotero://data URLs match
web API URLs, with a different prefix for the personal library (/library vs.
/users/12345)
- Miscellaneous fixes to data objects
Under the hood:
- Extensions now return an AsyncChannel, which is an nsIChannel implementation
that takes a promise-yielding generator that returns a string,
nsIAsyncInputStream, or file that will be used for the channel's data
- New function Zotero.Utilities.Internal.getAsyncInputStream() takes a
generator that yields either promises or strings and returns an async input
stream filled with the yielded strings
- Zotero.Router parsers URLs and extract parameters
- Zotero.Item.toResponseJSON()
Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
- Support for group libraries
- General support for multiple libraries of different types
- Streamlined sync support
- Using solely libraryID and key rather than itemID, and removed all itemID-changing code
- Combined two requests for increased performance and decreased server load
- Added warning on user account change
- Provide explicit error message on SSL failure
- Removed snapshot and link toolbar buttons and changed browser context menu options and drags to create parent items + snapshots
- Closes#786, Add numPages field
- Fixes#1063, Duplicate item with tags broken in Sync Preview
- Added better purging of deleted tags
- Added local user key before first sync
- Add clientDateModified to all objects for more flexibility in syncing
- Added new triples-based Relation object type, currently used to store links between items copied between local and group libraries
- Updated zotero.org translator for groups
- Additional trigger-based consistency checks
- Fixed broken URL drag in Firefox 3.5
- Disabled zeroconf menu option (no longer functional)
Developer-specific changes:
- Overhauled data layer
- Data object constructors no longer take arguments (return to 1.0-like API)
- Existing objects can be retrieved by setting id or library/key properties
- id/library/key must be set for new objects before other fields
- New methods:
- ZoteroPane.getSelectedLibraryID()
- ZoteroPane.getSelectedGroup(asID)
- ZoteroPane.addItemFromDocument(doc, itemType, saveSnapshot)
- ZoteroPane.addItemFromURL(url, itemType)
- ZoteroPane.canEdit()
- Zotero.CollectionTreeView.selectLibrary(libraryID)
- New Zotero.URI methods
- Changed methods
- Many data object methods now take a libraryID
- ZoteroPane.addAttachmentFromPage(link, itemID)
- Removed saveItem and saveAttachments parameters from Zotero.Translate constructor
- translate() now takes a libraryID, null for local library, or false to not save items (previously on constructor)
- saveAttachments is now a translate() parameter
- Zotero.flattenArguments() better handles passed objects
- Zotero.File.getFileHash() (not currently used)
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
