20369f41b3- Move commonly used scraper functions to ingester.js, rather than re-defining them in each scraper. This breaks Piggy Bank compatibility in our scrapers, but we will still be able to export our scrapers in a Piggy Bank compatible form. - Better handling of scraper RDF to item mapping. - Improved date handling. All scrapers now return ISO-style dates when possible.
Simon Kornblith
2006-06-18 19:04:32 +0000
3d881eec13- Make scrapers return standard ISO-style YYYY-MM-DD dates. Still need to work on journal article scrapers. - Ingester lets callback function save items, rather than saving them itself. - Better handling of multiple items in API, although no scrapers currently implement this.
Simon Kornblith
2006-06-17 21:21:15 +0000
953b1f9d20Lots of little things: - Fix item modify notify() on 1st row. - Ensure that new items are visible when added. - New functionality for creating new items (prevents a lot of problems). - Number-based fields display properly. - Fixed bug when creating and saving the first notes on an item. - New notes won't save empty.
David Norton
2006-06-17 00:57:50 +0000
32ce0da44aRestore the statement.reset() in valueQuery() that I accidentally deleted in r216
Dan Stillman
2006-06-16 22:17:46 +0000
bfa25cae37Display SQLite error in exceptions thrown from commitTransaction() and rollbackTransaction() (like the ones on field value updates that I'm trying to figure out at the moment)
Dan Stillman
2006-06-16 22:15:07 +0000
e8ed4281f9Change the DB query functions to return values of the actual types they are, rather than to return everything as strings -- this will prevent the need for parseInt on COUNT(*) values, etc. and is generally better
Dan Stillman
2006-06-16 21:44:41 +0000
936ef86584Fix Item.numNotes() and Item.getNotes() to work on items not yet in the DB (return 0 and [], respectively)
Dan Stillman
2006-06-16 21:33:03 +0000
bf0626cb58[interface] All editing elements that look like labels but provide functionality now offer hover feedback [fix] You shouldn't lose your changes if you select another item in the middle of editing a field. [fix] The dropdown menu to select notes doesn't steal the focus
David Norton
2006-06-16 17:21:30 +0000
380584986a[fix] If you do a sort, modify a note, etc. it does not reload the selected item [interface/fix] There will always be a secondary sort on date modified. (fixes the infamous Turkle Toggle issue once and for all)
David Norton
2006-06-16 16:30:03 +0000
4904c04e3eAdd dateCreated and dateModified columns to itemNotes
Dan Stillman
2006-06-16 16:09:18 +0000
3f27a12b0fTrigger notify('modify', 'item', noteID) on note modify
Dan Stillman
2006-06-16 15:57:52 +0000
942c6d5da2[interface] Number of notes column in items tree
David Norton
2006-06-16 15:27:22 +0000
08a570a38aItem.numNotes()
Dan Stillman
2006-06-16 15:18:01 +0000
3cdcf3093a[interface] Confirms delete of note [fix] Minor fix - no more exception on ItemTreeView when you hit the left or right arrow [ignore] simplified some checking in the note code.
David Norton
2006-06-16 15:15:42 +0000
8ff6c48001[interface] Custom textbox binding: multiline and timed together. [interface] Multiple notes: works like a charm
David Norton
2006-06-16 14:39:18 +0000
47d59f2dc1Add note contents to search
Dan Stillman
2006-06-16 08:04:01 +0000
7ef16e0a38Reverted sample data to remove notes field (independent notes will need a field for that but we'll deal with that later)
Dan Stillman
2006-06-16 07:56:24 +0000
7a8ddb1bebFix logic in _noteToTitle in itemPane.js (if no newline in note the title would be "Untitled Note") -- though it'd be better to do what Stickies do and just find the first bit of text, even if it's not on the first line (regex is probably easiest)
Dan Stillman
2006-06-16 07:44:55 +0000
fd85af40f8Many-to-one item note support in the schema and data layer -- still some issues on (I think) the interface side
Dan Stillman
2006-06-16 07:32:48 +0000
e417d8e690[interface] Although commented out, code in place to accept URL drags into collections (waiting on an Ingester.scrapeURL function) [interface] Multi-notes functionality (waiting on data layer) [docs] Major internal documentation written for itemTreeView.js and collectionTreeView.js (this actually does work ;-))
David Norton
2006-06-15 22:35:48 +0000
f635ee7788Make the retry interval 60 minutes, not 60 seconds
Dan Stillman
2006-06-15 21:24:04 +0000
8e97675cc9Added a timer to run repository checks while the browser is up
Dan Stillman
2006-06-15 21:06:24 +0000
2d20fe717dAdded Scholar.HTTP.browserIsOffline() and changed doGet() and doPost() to return false if so
Dan Stillman
2006-06-15 16:52:46 +0000
11a056ecd4_getDBVersion() caches the version number, so make sure _updateDBVersion() updates it
Dan Stillman
2006-06-15 16:28:11 +0000
70be7cf8fdSwitched Notes and Info tabs. The tabbox should always fill the allotted space. Hopefully?
David Norton
2006-06-14 17:43:02 +0000
8ed37732c1JS Date() takes months 0-11, for reasons that are far from clear
Dan Stillman
2006-06-14 16:59:29 +0000
2399f044e1[Drag and Drop] in the Collections Tree: Now checks to make sure that the correct type is being dragged, and that you aren't dropping a folder into subfolders, etc. [Drag and Drop] in Items Tree: You can drag items from one window into another, directly into the Items list. [Editing] Close the edit box and save when you click on its label
David Norton
2006-06-14 15:51:05 +0000
c8a74e96cdDon't be dumb
Dan Stillman
2006-06-14 15:41:25 +0000
3e11379c3aNotes now automatically save - no "save" button. The collections list does not resize randomly now. The pane on the right stays open all the time - even when 0/multiple items are selected. This is to avoid frequent resizing of the items pane. Temporarily, if the first "word" of a field's value is more than 29 characters long, it will set it to crop. This is for the long URLs, etc.
David Norton
2006-06-13 20:45:30 +0000
ba16889bb0Moved the Scholar.Hash constructor out of the main Scholar constructor, where I had put it for some reason -- ignore
Dan Stillman
2006-06-13 15:14:22 +0000
ca2045a305Scholar.Date.sqlToDate(string sqldate) -- function to convert SQL-formatted date (e.g. '2006-06-13 11:03:05' or '2006-06-13') into a JS date object
Dan Stillman
2006-06-13 15:07:08 +0000
39f9d2c3b3Wrapper functions for XMLHTTPRequest in scholar.js, stolen and adapted slightly from Simon's ingester code (those override the mime type to text/plain and only use responseText, I assume to conform to the Piggy Bank API, so unfortunately we can't use the same ones elsewhere)
Dan Stillman
2006-06-13 14:53:38 +0000
d42258b168Changed schema of scrapers table to use single GUID for scraperID
Dan Stillman
2006-06-12 15:43:24 +0000
37c0a61393[style] collection icons have padding
David Norton
2006-06-12 13:56:36 +0000
0b1765b549Move CreatorTypes constructor in code -- ignore
Dan Stillman
2006-06-12 13:08:36 +0000
c24c148e0bCache creator type names to prevent repeated DB lookups in CreatorTypes.getTypeName()
Dan Stillman
2006-06-12 13:05:30 +0000
5a045f5b50Cache item type names to prevent repeated DB lookups in ItemTypes.getTypeName()
Dan Stillman
2006-06-12 12:59:25 +0000
b1389bfc61[interface] Item Type column (will show up on the right if you already have Scholar installed, but on the left for all new installations.), sortable. [style] Better add/remove Creator buttons. [fix] The sorting should not randomly switch the order of two items with the same sort value (eg, Barnes vs. Barnes). [fix] The browser should not open with two sorted columns.
David Norton
2006-06-12 12:43:20 +0000
cebd6bde5b[interface] Images now appear on the collections tree and items tree. (yes, they are ugly).
David Norton
2006-06-09 16:36:18 +0000
68fed95e8fRenamed metadataPane.js/xul to match previous change.
David Norton
2006-06-09 15:54:44 +0000
fbcd247b09Cleaned up the code in the tree views. Fixed a bug on 'add' - items showing up in Library even if they were already there.
David Norton
2006-06-09 14:42:53 +0000
9f7437f5ca[interface] itemsTreeView correctly handles notify(erase) [data access] Collection.removeItem() calls trigger() with the right type
David Norton
2006-06-09 14:01:07 +0000
14ba1b8fbc[interface] collection stays selected on drag-and-drop. [interface] Collections notify() cleaned up a bit.
David Norton
2006-06-08 21:47:33 +0000
9daa0d3303Added 'notes' field to fields table and sample data (changes are less messy than they look--I just put 'notes' after 'rights' and 'source' and incremented the other fieldIDs)
Dan Stillman
2006-06-08 21:30:22 +0000
c3acdf04a6Don't mark item as changed if setField('') is called on an empty field (i.e. prevent unnecessary save() calls)
Dan Stillman
2006-06-08 21:08:50 +0000
3715f2c89e- Send a 'modify' to collections in addItem() and removeItem() when collections become or stop being empty
Dan Stillman
2006-06-08 20:41:37 +0000
368724b7de[interface] Collections should keep their open state on a drop.
David Norton
2006-06-08 20:28:50 +0000
7c9b1f785cOnly send itemView 'remove' notifictions for items if they're actually deleted from the DB, not if they're just removed from a collection
Dan Stillman
2006-06-08 20:02:26 +0000
01f50b9e4bOn Collection.changeParent(), change notify() action to 'move', and always send a three-value array in the order [collectionID, previousParent, newParent] -- 2nd or 3rd param will be null if moving from/to root
Dan Stillman
2006-06-08 19:58:54 +0000
1336842d74[interface] You should now be able to drop collections below the library (into root). [interface] New items are automatically selected.
David Norton
2006-06-08 19:34:58 +0000
4545a5d8a8Collection.hasDescendent(type, id) -- type is 'collection' or 'item', for now
Dan Stillman
2006-06-08 19:11:10 +0000
15ec35c927Reload collections (all, for now) on changeParent() to refresh hasChildCollections()
Dan Stillman
2006-06-08 18:53:49 +0000
1d14c69ad5[interface] Preliminary drag-and-drop. 1) Items into collections. 2) Collections into collections. Dan S, please check this out, I am getting some exceptions from the data access portion. [interface] Temporarily, "metadata" pane on the right is not resizable.
David Norton
2006-06-08 18:42:55 +0000
076ee0fad2Add PubMed scraper, fix a few other small bugs
Simon Kornblith
2006-06-08 01:26:40 +0000
393d19ca85Fix SQL error on middle creator remove
Dan Stillman
2006-06-08 00:16:35 +0000
f437917016Add Project MUSE scraper
Simon Kornblith
2006-06-07 21:26:55 +0000
cef0b19770Add TLC/YouSeeMore scraper
Simon Kornblith
2006-06-07 18:44:27 +0000
8fc4a02814[interface] You can resize the view/edit pane on the right. [interface] You can press 'escape' on editing a field to cancel changes. [style] The add/remove creator buttons provide rollover feedback.
David Norton
2006-06-07 18:20:45 +0000
1e48189c3bAdd SIRSI (old) scraper
Simon Kornblith
2006-06-07 17:44:55 +0000
07dad8fae9Add DRA, GEAC scrapers
Simon Kornblith
2006-06-07 16:48:03 +0000
eba82e2ac6ItemTreeView remembers the selection on notify(). MetadataPane editing updated a lot -- creator support added.
David Norton
2006-06-07 16:19:56 +0000
393807b152This isn't quite done (I'm discussing changing the scrapers schema with Simon to better handle scraper updates) but in the interest of getting the scrapers in for testing, I'll commit this now.
Dan Stillman
2006-06-07 15:27:21 +0000
882a96ee40Fixed brokenness on schema reinitialization
Dan Stillman
2006-06-07 14:35:04 +0000
448fde9ff1Made the schema update system moderately less convoluted
Dan Stillman
2006-06-07 01:02:59 +0000
098e90fea9Search field now searches only selection Library/Project. - Sort stays - Selection remembered
David Norton
2006-06-06 22:43:58 +0000
0753d78910- Add VLTS scraper - Fix loadDocument/processDocuments (broken by r145)
Simon Kornblith
2006-06-06 21:35:23 +0000
9bcaad5946Added optional second param to Items.search() to restrict search to a particular collectionID
Dan Stillman
2006-06-06 20:46:08 +0000
2513c829a6Simple, yet inefficient, searching from upper right-hand corner.
David Norton
2006-06-06 20:33:49 +0000
c807d335e9Changed Items.get() to return a regularly indexed array instead of one indexed by itemID--nothing actually used the keyed array, and this way it can be used for building trees directly (e.g. Scholar.Items.get(Scholar.Items.search('barnes')))
Dan Stillman
2006-06-06 20:15:50 +0000
b65a56b4d9Fairly major metadatapane reworking. It is now nextdoor to the items list. - Lots of work to be done. For example, the present way of showing a textbox is sort of a hack - taking a label, assigning certain properties to a textbox, then removing the label and placing the textbox in its place. I will be looking into our options. - Also, I need to figure out adding/editing/deleting creators. - When the textbox loses focus, it updates and saves the item (like iCal).
David Norton
2006-06-06 19:53:27 +0000
3aae8d3f89Fix Scholar.getItems() to not trigger an SQL query for every item
Dan Stillman
2006-06-06 19:06:40 +0000
152c9bf9e7- Small changes to MARC record support - Implemented loadDocument API, for loading and parsing the DOMs of HTML documents in the background - Added scraper code to SVN repository (now includes 12 scrapers, see Writeboard for details)
Simon Kornblith
2006-06-06 18:25:45 +0000
6c55e63eabAdd ' et al.' to firstCreator field when more than one creator
Dan Stillman
2006-06-06 17:37:52 +0000
60f7c8fccdScholar.Items.search(text) -- (extremely) simple fulltext search on all fields -- returns an array of ids of matching items
Dan Stillman
2006-06-06 08:02:29 +0000
9675ac9d01New function Scholar.Collection.changeParent([parentCollectionID]) -- move a collection into another collection in the DB
Dan Stillman
2006-06-05 21:58:01 +0000
b31be7af7ePut square brackets around ids in notifier debug output if multiple ids passed
Dan Stillman
2006-06-05 21:51:21 +0000
3796741911Changed all references of 'folder' to 'collection' in code to correspond to database / dataacess nomenclature.
David Norton
2006-06-05 18:01:53 +0000
560864ad6bConverted interface strings into localized entities/properties.
David Norton
2006-06-05 17:47:36 +0000
0a234c9f70Implemented Scholar.Date, a group of functions for handling date formatting, etc.
David Norton
2006-06-05 16:19:11 +0000
5411d4b67fSorting on a colum: Much, much, much, better. (hope)fully implemented!
David Norton
2006-06-05 15:49:11 +0000
56b1e37c13New items are properly displayed on a notify() New items are automatically selected.
David Norton
2006-06-05 13:56:59 +0000
adfcc5cd73Collection rename works (presently, using a somewhat awkward "rename collection" button)
David Norton
2006-06-05 13:08:16 +0000
c4d2e77b27Interface: Add Collections (debugged to view properly) Interface: Delete collection now reselects correctly.
David Norton
2006-06-05 11:58:16 +0000
d22d77fedfMore helpful exceptions in itemFields methods when given an invalid fieldID or a fieldID with no associated itemTypes (neither should actually happen once things are set up right)
Dan Stillman
2006-06-04 19:37:03 +0000
1d9fa85c15Missing association of ISSN with journalArticle type -- added to sample data (thanks Simon)
Dan Stillman
2006-06-04 18:53:22 +0000
85d8153024Add library, hooks for scraping MARC records.
Simon Kornblith
2006-06-03 22:26:01 +0000
bd840a4c58Collection names must be non-empty -- interface side might follow iTunes behavior ('untitled playlist') here
Dan Stillman
2006-06-03 20:51:48 +0000
e2c9ee0387- Collection.rename(name) -- renames collection in DB
Dan Stillman
2006-06-03 20:41:23 +0000
d01ff389d4Use the Hash datatype in Notifier to store observers and improve debug output to display how many we have registered when trigger() is called
Dan Stillman
2006-06-03 20:26:31 +0000
98f42001ccNew function Scholar.getRandomID( table, column [, max] ) for getting unique random keys in a DB table, because auto_increments make me feel dirty -- max is optional and defaults to 16383 (which should store in 2 bytes in SQLite), but increases automatically if a unique id can't be found after 10 tries
Dan Stillman
2006-06-03 20:23:19 +0000
bbbb243315Accept 'null':true as bound parameter type
Dan Stillman
2006-06-03 20:06:57 +0000
93652a137cFix issues with asynchronous scraping and XMLHttpRequest
Simon Kornblith
2006-06-02 23:53:42 +0000
c42991a5bfMetadata now is much more compact when not being edited. Close buttons on the Metadata and Notes.
David Norton
2006-06-02 23:05:24 +0000
c561ca129eReverting r111 change to bound parameters for Item.erase() (minus the exception fix) -- bound parameters/multiple statements/no can do
Dan Stillman
2006-06-02 21:11:14 +0000
338bb54fdcAdjust flattenArguments() to properly handle scalar values -- so now it can be passed pretty much anything and just return a flattened array
Dan Stillman
2006-06-02 20:51:34 +0000
013af36855Fixed some issues with object reloading -- because of the way JS closures work, simply overwriting the old object doesn't work, since the interface code retains a reference to the old object -- in other words, a memory leak and stale data. Existing item objects now reload in place instead. Same change made to collections, though I'll have to test that after I add Collection editing and save().
Dan Stillman
2006-06-02 19:48:22 +0000
bb57e6ba7dProvide visual feedback for scraping
Simon Kornblith
2006-06-02 18:22:34 +0000
8f34487205Fixed error in new item itemData insert (trying to bind across multiple statements) (yes we need unit tests)
Dan Stillman
2006-06-02 16:37:17 +0000
Simon Kornblith